From 320ea87b08a153a63903f4c01dbe362a4307634f Mon Sep 17 00:00:00 2001 From: "glm-5.3-flash" Date: Thu, 27 Aug 2026 14:19:24 +0000 Subject: [PATCH] docs: port architecture specs and ADRs from alknet-http; write new alkhttp ADRs 067-070 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 1 (SDD) — architecture documentation: Ported specs (adapted for alkcall, producer/consumer terms, 6-endpoint gateway, channels-over-WS, Sub/Pub operation types): - overview.md, http-server.md, http-adapters.md, http-mcp.md - README.md index (rewritten for alkhttp) New ADRs: - 067: WebSocket carries the channels protocol (8-byte chunk demux, channel 0 = alk/call, upgrade path /alk/channels) - 068: gateway /publish endpoint for Pub operations (NDJSON body) - 069: WebTransport out of scope in alkhttp (alknet concern) - 070: from_wss consumer adapter (wss feature, tokio-tungstenite) Ported ADRs (25, same numbers, port notes + amendments where the extraction changed facts): 001-004, 010, 014, 015, 017, 022, 023, 027, 034, 036, 037, 039, 041, 042, 044, 045, 046, 047, 048, 049, 051, 066. websocket.md rewritten for the channels session; open-questions.md seeded (OQ-01 WS byte-stream adapter, OQ-02 /publish framing, OQ-03 from_wss reconnect, OQ-04 browser client ownership). Verified: cargo test, clippy -D warnings, fmt, doc --no-deps. --- docs/architecture/README.md | 160 ++++ .../decisions/001-alpn-protocol-dispatch.md | 55 ++ .../decisions/002-protocol-handler-trait.md | 77 ++ .../decisions/003-crate-decomposition.md | 173 +++++ .../decisions/004-auth-as-shared-core.md | 91 +++ .../decisions/010-alpn-router-and-endpoint.md | 328 ++++++++ ...-material-flow-and-capability-injection.md | 240 ++++++ ...5-privilege-model-and-authority-context.md | 340 +++++++++ ...ll-protocol-client-and-adapter-contract.md | 587 +++++++++++++++ ...on-provenance-and-composition-authority.md | 705 +++++++++++++++++ .../decisions/023-operation-error-schemas.md | 479 ++++++++++++ ...dentity-redesign-acme-rawkey-decoupling.md | 374 +++++++++ ...outgoing-only-x509-and-three-peer-roles.md | 541 +++++++++++++ .../036-http-to-call-operation-mapping.md | 303 ++++++++ .../037-mcp-stdio-transport-exclusion.md | 204 +++++ ...9-http-server-and-client-host-colocated.md | 180 +++++ .../decisions/041-mcp-tool-gateway-pattern.md | 263 +++++++ .../decisions/042-openapi-gateway-pattern.md | 335 +++++++++ ...fer-webtransport-browsers-use-websocket.md | 555 ++++++++++++++ .../045-to-openapi-gateway-spec-versioning.md | 231 ++++++ .../046-assembly-layer-custom-http-routes.md | 268 +++++++ .../047-remove-direct-call-http-surface.md | 351 +++++++++ ...48-websocket-native-session-not-gateway.md | 408 ++++++++++ ...049-streaming-handler-for-subscriptions.md | 422 +++++++++++ .../051-yaml-input-for-from-openapi.md | 241 ++++++ .../066-from-jsonschema-as-http-adapter.md | 240 ++++++ .../067-websocket-carries-channels.md | 170 +++++ .../decisions/068-gateway-publish-endpoint.md | 136 ++++ .../069-webtransport-out-of-scope.md | 102 +++ .../070-from-wss-consumer-adapter.md | 137 ++++ docs/architecture/http-adapters.md | 711 ++++++++++++++++++ docs/architecture/http-mcp.md | 481 ++++++++++++ docs/architecture/http-server.md | 612 +++++++++++++++ docs/architecture/open-questions.md | 89 +++ docs/architecture/overview.md | 303 ++++++++ docs/architecture/websocket.md | 377 ++++++++++ src/adapters/mod.rs | 1 + src/client/mod.rs | 1 + src/gateway/mod.rs | 1 + src/lib.rs | 2 +- src/server/mod.rs | 1 + src/websocket/mod.rs | 1 + 42 files changed, 11275 insertions(+), 1 deletion(-) create mode 100644 docs/architecture/README.md create mode 100644 docs/architecture/decisions/001-alpn-protocol-dispatch.md create mode 100644 docs/architecture/decisions/002-protocol-handler-trait.md create mode 100644 docs/architecture/decisions/003-crate-decomposition.md create mode 100644 docs/architecture/decisions/004-auth-as-shared-core.md create mode 100644 docs/architecture/decisions/010-alpn-router-and-endpoint.md create mode 100644 docs/architecture/decisions/014-secret-material-flow-and-capability-injection.md create mode 100644 docs/architecture/decisions/015-privilege-model-and-authority-context.md create mode 100644 docs/architecture/decisions/017-call-protocol-client-and-adapter-contract.md create mode 100644 docs/architecture/decisions/022-handler-registration-provenance-and-composition-authority.md create mode 100644 docs/architecture/decisions/023-operation-error-schemas.md create mode 100644 docs/architecture/decisions/027-tls-identity-redesign-acme-rawkey-decoupling.md create mode 100644 docs/architecture/decisions/034-outgoing-only-x509-and-three-peer-roles.md create mode 100644 docs/architecture/decisions/036-http-to-call-operation-mapping.md create mode 100644 docs/architecture/decisions/037-mcp-stdio-transport-exclusion.md create mode 100644 docs/architecture/decisions/039-http-server-and-client-host-colocated.md create mode 100644 docs/architecture/decisions/041-mcp-tool-gateway-pattern.md create mode 100644 docs/architecture/decisions/042-openapi-gateway-pattern.md create mode 100644 docs/architecture/decisions/044-defer-webtransport-browsers-use-websocket.md create mode 100644 docs/architecture/decisions/045-to-openapi-gateway-spec-versioning.md create mode 100644 docs/architecture/decisions/046-assembly-layer-custom-http-routes.md create mode 100644 docs/architecture/decisions/047-remove-direct-call-http-surface.md create mode 100644 docs/architecture/decisions/048-websocket-native-session-not-gateway.md create mode 100644 docs/architecture/decisions/049-streaming-handler-for-subscriptions.md create mode 100644 docs/architecture/decisions/051-yaml-input-for-from-openapi.md create mode 100644 docs/architecture/decisions/066-from-jsonschema-as-http-adapter.md create mode 100644 docs/architecture/decisions/067-websocket-carries-channels.md create mode 100644 docs/architecture/decisions/068-gateway-publish-endpoint.md create mode 100644 docs/architecture/decisions/069-webtransport-out-of-scope.md create mode 100644 docs/architecture/decisions/070-from-wss-consumer-adapter.md create mode 100644 docs/architecture/http-adapters.md create mode 100644 docs/architecture/http-mcp.md create mode 100644 docs/architecture/http-server.md create mode 100644 docs/architecture/open-questions.md create mode 100644 docs/architecture/overview.md create mode 100644 docs/architecture/websocket.md diff --git a/docs/architecture/README.md b/docs/architecture/README.md new file mode 100644 index 0000000..7c93278 --- /dev/null +++ b/docs/architecture/README.md @@ -0,0 +1,160 @@ +--- +status: draft +last_updated: 2026-08-27 +--- + +# alkhttp + +HTTP interface for the alk stack: serves HTTP/1.1 and HTTP/2 on standard +ALPNs (with WebSocket upgrade carrying the channels protocol for browser +bidirectional access to the call protocol), and hosts the HTTP-backed +call-protocol adapters (`from_openapi`, `from_jsonschema`, `from_mcp`, +`to_openapi`, `to_mcp`, `from_wss`). HTTP/3 + WebTransport (`h3`) is +**out of scope** per [ADR-069](decisions/069-webtransport-out-of-scope.md) — +it is an alknet-side concern, not an alkhttp one. + +alkhttp is the extraction of `alknet-http` from the alknet mono-repo, +re-implemented on the published crates: `alkvault` (secrets) and +`alkcall` 0.1.1 (call protocol + channels protocol + vendored core +types — the former `alknet-core` and `alknet-call` merged). + +## Documents + +| Document | Status | Description | +|----------|--------|-------------| +| [overview.md](overview.md) | draft | Crate purpose, two roles (server + client host), dependency edges, adapter location map | +| [http-server.md](http-server.md) | draft | `HttpAdapter` (`ProtocolHandler` for `h2`/`http/1.1` + WS upgrade route), axum over a `BiStream`, Bearer auth, stealth, `/healthz`; WS hands off to the channels session spec | +| [websocket.md](websocket.md) | draft | WebSocket browser bidirectional path — the WS connection carries the **channels protocol** (8-byte chunk multiplexing, channel 0 pre-negotiated as `alk/call`); framing via the WS↔byte-stream adapter, dispatch, bidirectionality, connection-local Layer 2 overlay, browsers-are-not-peers rationale, streaming (native, no SSE), `from_wss` consumer adapter | +| [http-adapters.md](http-adapters.md) | draft | `from_openapi` (reqwest client; JSON + YAML input per ADR-051), `from_jsonschema` (single-endpoint reqwest forwarding handler per ADR-066), `to_openapi` (OpenAPI projection of the 6-endpoint gateway), `from_wss` (WSS consumer adapter per ADR-070); no-env-vars invariant point | +| [http-mcp.md](http-mcp.md) | draft | `from_mcp` / `to_mcp` (feature-gated), streamable-HTTP-only, stdio exclusion | + +## Applicable ADRs + +### Ported from alknet (same numbers) + +| ADR | Title | Relevance | +|-----|-------|-----------| +| [001](decisions/001-alpn-protocol-dispatch.md) | ALPN-Based Protocol Dispatch | `HttpAdapter` registers on standard HTTP ALPNs | +| [002](decisions/002-protocol-handler-trait.md) | ProtocolHandler Trait | `HttpAdapter` implements `ProtocolHandler` | +| [003](decisions/003-crate-decomposition.md) | Crate Decomposition | alkhttp depends on alkcall alone (protocol-foundation exception, Amendment 1) | +| [004](decisions/004-auth-as-shared-core.md) | Auth as Shared Core | Bearer → `resolve_from_token` | +| [010](decisions/010-alpn-router-and-endpoint.md) | ALPN Router and Endpoint | Stealth mode = HTTP handler on standard ALPNs | +| [014](decisions/014-secret-material-flow-and-capability-injection.md) | Secret Material Flow | `from_openapi`/`from_mcp` are the credential injection point | +| [015](decisions/015-privilege-model-and-authority-context.md) | Privilege Model | Adapter-registered ops are `Internal` by default | +| [017](decisions/017-call-protocol-client-and-adapter-contract.md) | Call Protocol Client and Adapter Contract | `OperationAdapter` trait; `to_*` are projections; published-spec contract | +| [022](decisions/022-handler-registration-provenance-and-composition-authority.md) | Handler Registration, Provenance, Composition Authority | `from_openapi`/`from_mcp`/`from_jsonschema` produce leaf bundles (`FromJsonSchema` handler-bearing per ADR-066) | +| [023](decisions/023-operation-error-schemas.md) | Operation Error Schemas | `from_openapi`/`from_jsonschema`/`to_openapi` error fidelity; `HTTP_` error codes | +| [027](decisions/027-tls-identity-redesign-acme-rawkey-decoupling.md) | TLS Identity Redesign | Browsers require X.509; applies to any browser-facing TLS (TLS provisioning itself is an alknet concern) | +| [034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) | Outgoing-Only X.509 and Three Peer Roles | Browsers are not alknet peers (§4) | +| [036](decisions/036-http-to-call-operation-mapping.md) | HTTP-to-Call Operation Mapping | ~~Direct path mapping~~ — **routing superseded by ADR-047**; non-routing clauses survive | +| [037](decisions/037-mcp-stdio-transport-exclusion.md) | MCP Stdio Transport Exclusion | Streamable HTTP only; stdio not built | +| [039](decisions/039-http-server-and-client-host-colocated.md) | HTTP Server and Client Host Colocated | One crate for server + client host (shared HTTP deps, shared mapping) | +| [041](decisions/041-mcp-tool-gateway-pattern.md) | MCP Tool-Gateway Pattern for to_mcp | 4 fixed gateway tools (search/schema/call/batch); Sub AND Pub excluded | +| [042](decisions/042-openapi-gateway-pattern.md) | OpenAPI Gateway Pattern for to_openapi | Fixed gateway endpoints, not one path per operation; per-caller AccessControl-filtered | +| [044](decisions/044-defer-webtransport-browsers-use-websocket.md) | Defer h3/WebTransport; Browsers Use WebSocket | WS as browser path stands; deferral mechanics superseded by ADR-069 (removal) | +| [045](decisions/045-to-openapi-gateway-spec-versioning.md) | to_openapi Gateway-Spec Versioning | `info.version` (semver) tracks the gateway endpoint contract | +| [046](decisions/046-assembly-layer-custom-http-routes.md) | Assembly-Layer Custom HTTP Routes | `extra_routes: Option` at construction | +| [047](decisions/047-remove-direct-call-http-surface.md) | Remove the Direct-Call HTTP Surface | The 6 gateway endpoints are the sole invoke path | +| [048](decisions/048-websocket-native-session-not-gateway.md) | WebSocket Carries the Native Session, Not the Gateway Shape | Amended by ADR-067: WS carries the channels session; channel 0 carries the native call session | +| [049](decisions/049-streaming-handler-for-subscriptions.md) | Streaming Handler for Subscription Operations | `HandlerKind::Stream` for Sub; `HandlerKind::Sink` for Pub (alkcall ADR-046) | +| [051](decisions/051-yaml-input-for-from-openapi.md) | YAML Input Format for from_openapi | `from_json`/`from_yaml`/`from_str`; JSON-first detection; `yaml_serde` | +| [066](decisions/066-from-jsonschema-as-http-adapter.md) | `from_jsonschema` as HTTP-Backed Single-Endpoint Adapter | Real reqwest-backed single-endpoint adapter; provenance stays in alkcall | + +### New in alkhttp + +| ADR | Title | Summary | +|-----|-------|---------| +| [067](decisions/067-websocket-carries-channels.md) | WebSocket Carries the Channels Protocol | WS = in-line channels substrate: 8-byte chunk demux, channel 0 pre-negotiated as `alk/call`, upgrade path `/alk/channels`; the shared `Dispatcher` runs on channel 0 | +| [068](decisions/068-gateway-publish-endpoint.md) | Gateway `/publish` Endpoint | 6th gateway endpoint for `OperationType::Pub` (producer→consumer streaming); NDJSON request body → `call.published` chunks | +| [069](decisions/069-webtransport-out-of-scope.md) | WebTransport Out of Scope | h3/WebTransport removed from alkhttp scope entirely (an alknet concern); supersedes the deferral framing of ADR-044 | +| [070](decisions/070-from-wss-consumer-adapter.md) | `from_wss` Consumer Adapter | Import a remote node's operations over WSS — same-protocol importer, channels-over-WS as transport; `wss` feature gate | + +## Relevant Open Questions + +Open questions are tracked in [open-questions.md](open-questions.md). +Key ones: + +| OQ | Title | Status | Relevance | +|----|-------|--------|-----------| +| OQ-01 | WS message ↔ byte-stream adaptation | open | The channels demux reads bytes (`read_exact`); axum WS is message-oriented. The WS↔byte-stream adapter's buffering/flush semantics are the core implementation risk for both the server path and `from_wss` | +| OQ-02 | `/publish` body framing details | open | NDJSON line = one published chunk; error envelope position (final JSON body vs HTTP trailer) needs a concrete decision before the gateway-spec version bumps | +| OQ-03 | `from_wss` reconnection semantics | open | Does a dropped WSS connection re-discover (`services/list`) and re-register, or hold stale registrations? | + +## Key Design Principles + +1. **HTTP is both a server surface and a client transport for adapters.** + Inbound HTTP (`h2`/`http/1.1` + WebSocket upgrade) is served by `axum` + over a `BiStream`; outbound HTTP (`from_openapi`/`from_mcp` + forwarding) uses `reqwest`. Both directions share the same HTTP + dependencies, which is why they live in one crate. See + [ADR-039](decisions/039-http-server-and-client-host-colocated.md). +2. **The HTTP surface is a fixed-endpoint gateway — 6 endpoints, not a + per-operation REST tree.** An HTTP client invokes an operation via + `POST /call` with `{ "operation": "/fs/readFile", "input": {...} }`, + discovers what it can call via `AccessControl`-filtered `GET /search`, + learns an operation's shape via `GET /schema`, streams a `Sub` + operation via `POST /subscribe` (SSE), and feeds a `Pub` operation via + `POST /publish` (NDJSON body). There is no per-operation + `POST /{service}/{op}` direct-call surface (removed by ADR-047; the + per-caller API surface is the default). `to_openapi` *describes* this + gateway surface. A deployment that wants a REST-like per-operation + surface builds it as a custom route projection (ADR-046). See + [ADR-042](decisions/042-openapi-gateway-pattern.md) and + [ADR-047](decisions/047-remove-direct-call-http-surface.md). +3. **Standard ALPNs, not alk ALPNs.** `h2`, `http/1.1` are + IANA-registered ALPN strings. Any HTTP client (browser, curl, axios) + connects without knowing about the alk stack — the TLS handshake + negotiates `h2` or `http/1.1` normally. This is the stealth mapping + ([ADR-010](decisions/010-alpn-router-and-endpoint.md)). +4. **`from_openapi`/`from_mcp`/`from_jsonschema` are the no-env-vars + injection point.** The forwarding handlers read + `context.capabilities`, not `std::env::var`. See + [ADR-014](decisions/014-secret-material-flow-and-capability-injection.md). +5. **MCP streamable HTTP only; stdio is not built.** stdio = spawn + arbitrary executable = RCE. See + [ADR-037](decisions/037-mcp-stdio-transport-exclusion.md). +6. **WebSocket is the browser bidirectional path, and it carries the + channels protocol.** A browser upgrades an HTTP/1.1 or HTTP/2 request + to WebSocket and speaks the channels protocol over binary WS messages: + the 8-byte chunk header multiplexes N channels; channel 0 is + pre-negotiated as `alk/call` and carries the native call-protocol + session ([ADR-067](decisions/067-websocket-carries-channels.md)). + Both sides can initiate calls on channel 0; the browser may open + further channels via the per-ALPN open ops, exactly as a Rust peer + would over an in-line transport. The 6 gateway endpoints are the HTTP + one-directional projection and **do not appear on the WS path** — + discovery is via `services/list`/`services/schema` as call-protocol + ops ([ADR-048](decisions/048-websocket-native-session-not-gateway.md)). + This supersedes the alknet design where WS carried bare + `EventEnvelope` messages (one envelope per WS message). +7. **Browsers are not alk peers.** A browser over WebSocket + authenticates by bearer token, gets no `PeerId`, and its registered + ops land in a connection-local Layer 2 overlay. See + [ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) §4. +8. **Producer/consumer, not server/client.** Both sides of a call or + channels connection can initiate. A producer exposes operations + (call) or opens data channels; a consumer calls operations or opens + channels; both sides can be both simultaneously. Connection direction + (who opened it) is independent of call/channel direction (who + calls/opens). The `from_*`/`to_*` adapter names remain directional + because OpenAPI and MCP are client/server protocols — that + directionality is a property of those protocols, not of the call + protocol. + +## References + +- The alkcall crate (`/workspace/@alkdev/alkcall`, + crates.io `alkcall 0.1.1`) — call protocol, channels protocol, + vendored core types. Its `docs/architecture/` owns the call-protocol + and channels decisions cited throughout: wire format (ADR-014), stream + model (ADR-015), registry layering (ADR-019), abort cascade (ADR-020), + streaming handler (ADR-021), adapter contract (ADR-022), channels wire + format (ADR-034/035), channel 0 pre-negotiation (ADR-036), + ChannelClient (ADR-043), Pub/Sink (ADR-046), openable ALPNs as + operations (ADR-047). +- The alkvault crate (`/workspace/@alkdev/alkvault`, crates.io) — + secrets; feeds `Capabilities` at the assembly layer (no direct + dependency from alkhttp). +- The alknet mono-repo (`/workspace/@alkdev/alknet`) — the source of + this extraction; retains the endpoint/transport/peer-graph concerns + (dial, TLS, QUIC, WebTransport) and the ADRs for them. \ No newline at end of file diff --git a/docs/architecture/decisions/001-alpn-protocol-dispatch.md b/docs/architecture/decisions/001-alpn-protocol-dispatch.md new file mode 100644 index 0000000..6c28761 --- /dev/null +++ b/docs/architecture/decisions/001-alpn-protocol-dispatch.md @@ -0,0 +1,55 @@ +# ADR-001: ALPN-Based Protocol Dispatch + +*Ported from alknet ADR-001 (ALPN-Based Protocol Dispatch); re-targeted to alkhttp.* + +## Status + +Accepted + +## Context + +The previous architecture used a three-layer model: transports produced byte streams, interfaces defined how to interpret those streams (StreamInterface, MessageInterface), and OperationEnv dispatched operations through local, irpc, or remote paths. This required a ListenerConfig enum with three variants (Stream, Http, Dns), a server accept loop handling three different listener types, and a complex dispatch model that mixed concerns across layers. + +Protocol detection was done by byte-peeking — the server read the first bytes of an incoming connection and guessed which protocol the client was speaking. This is fragile, limits protocol extensibility, and cannot work with encrypted transports where the payload is opaque. + +ALPN (Application-Layer Protocol Negotiation) is a TLS extension where the client advertises supported protocols during the handshake and the server selects one. QUIC builds on this natively — every QUIC connection has an ALPN. This is the same pattern iroh uses: `Router` dispatches incoming QUIC connections to `ProtocolHandler` implementations based on the ALPN string. Hickory DNS registers ALPN protocols (`dot`, `doq`, `h2`, `h3`). The reverse-proxy project at `@alkdev/reverse-proxy` uses the same pattern for TLS. + +The core insight: **a service IS an ALPN**. Every protocol handler registers an ALPN string on a shared endpoint. The ALPN negotiation during the handshake routes the connection to the correct handler before any application bytes are read. + +## Decision + +All protocol dispatch in alkhttp is ALPN-based: the `HttpAdapter` registers on the standard HTTP ALPNs (`h2`, `http/1.1`) on the shared endpoint (the endpoint and TLS/transport layer are alkcall/alknet-side concerns — see the alkcall crate docs). A single endpoint accepts connections, and the ALPN string selected during the handshake determines which `ProtocolHandler` receives the connection. There is no byte-peeking, no ListenerConfig enum, and no three-layer dispatch model. + +The endpoint advertises the union of all registered handlers' ALPN strings. When a client connects, the TLS/QUIC handshake negotiates the ALPN. If the client's offered ALPNs and the server's advertised ALPNs have no intersection, the handshake fails — this is the correct behavior, not an error to work around. + +## Consequences + +**Positive:** +- Single dispatch mechanism replaces three separate listener types +- Protocol detection happens at the TLS layer, not application layer — no byte-peeking +- Adding a new protocol is registering a new ALPN string — no server code changes +- Each handler owns its entire wire format — no shared framing layer +- QUIC connections are cheap — a client that needs multiple protocols opens one connection per ALPN, all multiplexed over the same UDP flow +- Stealth mode (byte-peek protocol detection on port 443) is unnecessary — ALPN negotiation handles this cleanly +- WASM story is clean: handlers receive byte streams, protocol parsers that operate on bytes compile to WASM + +**Negative:** +- ALPN is negotiated per-connection, not per-stream — a client that wants to use multiple ALPNs (e.g., SSH and call protocol) opens separate QUIC connections for each. QUIC connections are cheap (multiplexed over the same UDP flow), so this is acceptable, but it means `alkcall` cannot serve as a multiplexer for other ALPNs within a single connection unless explicitly designed to do so (see ADR-006, ALPN convention and connection model, in the alkcall crate docs). +- All protocols must be registered at endpoint creation time (or use hot-reload via ArcSwap for dynamic addition) +- Custom protocols require reserving ALPN strings — we own the `alknet/` namespace +- Debugging requires knowing which ALPN was negotiated (mitigated by logging at the endpoint level) + +## References + +- Pivot proposal (alknet mono-repo): `docs/research/pivot/alpn-service-architecture.md` +- [ADR-002](002-protocol-handler-trait.md): ProtocolHandler trait +- [ADR-003](003-crate-decomposition.md): Crate decomposition +- iroh reference (alknet mono-repo): `docs/research/references/iroh/` (ALPN dispatch, ProtocolHandler pattern) +- Replaces the old three-layer model (StreamInterface/MessageInterface/OperationEnv) + +## Port notes + +- Decision retargeted from alknet-wide dispatch to alkhttp: the `HttpAdapter` registers on the standard HTTP ALPNs (`h2`, `http/1.1`); the shared endpoint/TLS-transport layer is an alkcall/alknet-side concern, so "a single QUIC+TLS endpoint" became the transport-agnostic "a single endpoint" (and "shared QUIC+TLS endpoint" → "shared endpoint" in the Context insight). +- The ADR does not list h3 as an alkhttp deferred registration; the only h3 mention is the Hickory DNS example of ALPN registration, left as-is. h3/WebTransport is an alknet-side concern, not an alkhttp one. +- ADR-006 reference converted to a textual "alkcall crate docs" reference (the connection/ALPN-convention ADRs are alkcall-internal and not ported here). +- Mono-repo-relative research paths (`docs/research/...`) annotated as alknet mono-repo paths. \ No newline at end of file diff --git a/docs/architecture/decisions/002-protocol-handler-trait.md b/docs/architecture/decisions/002-protocol-handler-trait.md new file mode 100644 index 0000000..44f7af7 --- /dev/null +++ b/docs/architecture/decisions/002-protocol-handler-trait.md @@ -0,0 +1,77 @@ +# ADR-002: ProtocolHandler Trait + +*Ported from alknet ADR-002 (ProtocolHandler Trait); re-targeted to alkhttp.* + +## Status + +Accepted + +## Context + +The previous architecture had two separate interface traits: `StreamInterface` (for byte-stream protocols like SSH, raw TCP) and `MessageInterface` (for message-based protocols like DNS, HTTP). This split created complexity — each interface type needed its own listener configuration, its own dispatch path, and its own framing assumptions. The `ListenerConfig` enum had three variants. The server accept loop handled three different listener types. + +In practice, the distinction between "stream" and "message" protocols is artificial at the handler level. SSH starts as a byte stream but internally multiplexes channels and messages. DNS over QUIC is message-based but arrives as a stream of frames. HTTP/2 is both — bidirectional streams with message semantics. Every protocol can be modeled as "receive a byte stream, manage your own wire format." + +iroh's `ProtocolHandler` trait demonstrates this: it takes a bidirectional stream and the handler is responsible for its own protocol. One trait, one dispatch point. + +## Decision + +A single `ProtocolHandler` trait replaces both `StreamInterface` and `MessageInterface`: + +> **Note**: The signature below was revised by ADR-007. The `handle()` method +> now receives a `Connection` (not a `BiStream`) — see ADR-007 for the +> current authoritative signature (see the alkcall crate docs). The original +> signature is retained here for historical context. + +```rust +#[async_trait] +pub trait ProtocolHandler: Send + Sync + 'static { + /// The ALPN string this handler claims (e.g. b"alknet/ssh") + fn alpn(&self) -> &'static [u8]; + + /// Handle an incoming connection (revised by ADR-007 to receive + /// `Connection` instead of `BiStream`) + async fn handle(&self, connection: Connection, auth: &AuthContext) -> Result<(), HandlerError>; +} +``` + +- `alpn()` returns a static byte string — the handler's ALPN identifier +- `handle()` receives a `Connection` (revised by ADR-007 from the original + `BiStream`) and an `AuthContext` carrying the authenticated identity, and + returns `HandlerError` on failure +- Every handler manages its own wire format — no shared framing, no StreamInterface/MessageInterface split +- The `ListenerConfig` enum is eliminated — ALPN advertisement configuration replaces it + +**AuthContext resolution is hybrid** (see ADR-004, OQ-02 resolution): the endpoint resolves what it can before calling `handle()` (e.g., TLS client certificate fingerprint), and the handler resolves what it must inside `handle()` (e.g., AuthToken in the first frame of a call stream). The `AuthContext` passed to `handle()` may contain partial identity information — the handler is responsible for completing authentication if the endpoint didn't have enough information. + +In alkhttp, the `HttpAdapter` implements this trait on the standard HTTP ALPNs (`h2`, `http/1.1`): it accepts one bidirectional stream (BiStream) yielded by `Connection::accept_bi()`, serves the HTTP/1.1 + HTTP/2 surface over it, and extracts the Bearer credential for auth resolution. + +## Consequences + +**Positive:** +- One trait, one dispatch point — eliminates the StreamInterface/MessageInterface split and ListenerConfig enum +- Each handler owns its wire format — no shared framing assumptions that constrain protocol design +- Adding a new protocol is implementing one trait with two methods +- Testable in isolation — give a handler a mock BiStream and AuthContext +- WASM-compatible in principle — handlers that don't need tokio runtime features compile to WASM + +**Negative:** +- Every handler must implement its own framing — no shared "read a length-prefixed message" utility (mitigated: common utilities can live in `alkcall` without mandating their use) +- Handlers that want message semantics must build them (mitigated: the call protocol provides this as a handler, not a mandatory layer) +- AuthContext resolution is hybrid — the endpoint resolves what it can (TLS-level auth), but handlers that need protocol-level credential extraction must do so inside handle(). This means AuthContext may be partial when handle() is called. Handlers must not assume AuthContext is fully resolved. + +## References + +- Pivot proposal (alknet mono-repo): `docs/research/pivot/alpn-service-architecture.md` +- [ADR-001](001-alpn-protocol-dispatch.md): ALPN-based protocol dispatch +- [ADR-004](004-auth-as-shared-core.md): Auth as shared core (IdentityProvider) +- ADR-007: BiStream type definition — revised this ADR's signature from BiStream to Connection (see the alkcall crate docs) +- iroh ProtocolHandler pattern (alknet mono-repo): `docs/research/references/iroh/` +- Replaces StreamInterface, MessageInterface, and ListenerConfig + +## Port notes + +- Framing layer renames: "common utilities can live in alknet-core" → `alkcall`; "alknet-call provides this as a handler" → "the call protocol provides this as a handler". +- iroh phrasing made transport-agnostic: "takes a bidirectional QUIC stream" → "takes a bidirectional stream". +- Added one clause pinning the trait to alkhttp's use: `HttpAdapter` implements it on `h2`/`http/1.1` and serves HTTP over the BiStream from `Connection::accept_bi()`. +- ADR-007 references (BiStream type definition, alkcall-internal) converted to textual "alkcall crate docs" references. \ No newline at end of file diff --git a/docs/architecture/decisions/003-crate-decomposition.md b/docs/architecture/decisions/003-crate-decomposition.md new file mode 100644 index 0000000..30ba598 --- /dev/null +++ b/docs/architecture/decisions/003-crate-decomposition.md @@ -0,0 +1,173 @@ +# ADR-003: Crate Decomposition + +*Ported from alknet ADR-003 (Crate Decomposition); re-targeted to alkhttp.* + +## Status + +Accepted + +## Context + +The previous architecture had a monolithic core crate containing transport, interface, server, client, call, auth, config, socks5, credentials, and HTTP — all in one crate with interdependent modules. This created coupling (interface types depended on auth, server depended on call, everything depended on config) and made it impossible to use individual components independently. + +The ALPN dispatch model eliminates the need for a shared interface layer. Each handler is self-contained — it receives a byte stream and manages its own protocol. This naturally decomposes into separate crates. + +Key constraints: +- Protocol crates must depend on the shared core for auth/identity/config — but not on each other +- The vault crate (alkvault) is already standalone (no core dependency) and must remain so (see ADR-008) +- The CLI binary assembles everything — it's the only crate that depends on all handler crates +- Handlers with protocol-agnostic cores (SFTP, call protocol) preserve the WASM door — browser clients can implement the wire format over WebTransport (see ADR-009, ADR-013) +- The call crate includes the call protocol client and adapter traits, not just the server side — this enables agent and NAPI consumers to use it for remote invocation +- Rust is the canonical implementation language. TypeScript is a reference/browser adaptation, not a parallel implementation (see ADR-013) + +## Decision + +The alknet workspace decomposed into the following crates (now extracted and +published as alkvault, alkcall, and the per-protocol crates; alkhttp is the +HTTP interface crate): + +| Crate | Responsibility | Depends on | +|-------|---------------|------------| +| shared core (now `alkcall`, which vendors the core types) | ProtocolHandler trait, ALPN router, endpoint, BiStream, AuthContext, IdentityProvider, config, ArcSwap dynamic config | tokio, quinn, rustls, iroh (feature-gated, added by ADR-010) | +| `alkvault` | Local key vault: BIP39/SLIP-0010/AES-GCM key derivation, encryption | (standalone, no core dependency) | +| `alknet-ssh` | SshAdapter (russh, SOCKS5, port forwarding) | core, russh | +| call crate (now `alkcall`) | CallAdapter (JSON-RPC via hand-rolled EventEnvelope framing, operation registry, pub/sub, access control, call protocol client, adapter traits) | core | +| `alknet-agent` | Agent service: LLM execution loop (forked aisdk), tool dispatch via call protocol, provider key retrieval via vault | call | +| `alknet-git` | GitAdapter (gix, pkt-line protocol) | core, gix | +| `alknet-sftp` | SftpAdapter (russh-sftp protocol core) | core, russh-sftp | +| `alknet-msg` | MessageAdapter (E2E encryption, mixnet) | core | +| `alkhttp` | HttpAdapter (axum, REST API, MCP endpoint) | alkcall, axum | +| `alknet-dns` | DnsAdapter (hickory-proto, pkarr, service discovery) | core, hickory-proto | +| `alknet-napi` | Node.js native addon — thin NAPI projection of the call protocol client | call, napi-rs | +| `alknet` | CLI binary — registers handlers, starts endpoint | all handler crates, alkvault | + +Dependency flow: +``` +alkvault (standalone) +core ← all handler crates ← alknet (CLI) +call ← alknet-agent +call ← alknet-napi +``` + +No handler crate depends on another handler crate. Cross-handler communication goes through the call protocol (alkcall) or through the core's endpoint. + +alknet-agent depends on the call crate (not the core directly) because it uses the call protocol client for tool dispatch and the operation registry for tool registration. It receives LLM provider keys through capabilities injected at the assembly layer (from alkvault), never from environment variables and never over the call protocol. See ADR-008 and ADR-014. + +alknet-napi is a thin projection layer — it exposes the Rust call protocol client to Node.js via NAPI. It does not contain business logic or adapter implementations. See ADR-013. + +## Consequences + +**Positive:** +- Each handler can be developed, tested, and versioned independently +- WASM-compatible handlers (sftp, call) don't pull in heavy dependencies (russh, axum) +- alkvault remains standalone — no circular dependency risk +- New handlers are added by creating a crate and registering it with the endpoint +- Clean separation of concerns — each crate has one job + +**Negative:** +- More crates to manage in the workspace — workspace Cargo.toml and version coordination +- Shared types (AuthContext, BiStream) must live in the core crate — if they change, all handlers recompile +- The CLI binary has a large dependency tree (all handlers) — but this is expected for a binary that assembles everything +- Testing cross-handler behavior requires integration tests in the CLI or a test utility crate + +## References + +- Pivot proposal (alknet mono-repo): `docs/research/pivot/alpn-service-architecture.md` +- [ADR-001](001-alpn-protocol-dispatch.md): ALPN-based protocol dispatch +- [ADR-002](002-protocol-handler-trait.md): ProtocolHandler trait +- [ADR-004](004-auth-as-shared-core.md): Auth as shared core (IdentityProvider) +- ADR-005: irpc as call protocol foundation (superseded by ADR-064) + +## Amendments + +### Amendment 1 (2026-06-29): the call crate is a protocol-foundation crate + +The Decision table lists the call crate as a handler crate that "depends +on the core, irpc." The dependency-flow diagram and the "No handler +crate depends on another handler crate" rule were written before +the HTTP crate (which implements `from_openapi`/`from_mcp`/`to_openapi`/ +`to_mcp` and therefore needs the call crate's `OperationSpec`, `Handler`, +`HandlerRegistration`, and `OperationAdapter` trait) was specced. + +**Clarification:** the call crate is both a handler crate (it implements +`ProtocolHandler` on ALPN `alknet/call`) *and* the protocol-foundation +crate that alknet-agent, alknet-napi, and the HTTP crate consume for +the operation registry, adapter contract, and call client. The "no +handler crate depends on another handler crate" rule applies to peer +handler crates (e.g., alkhttp does not depend on `alknet-ssh`); +the call crate is a protocol-foundation crate in the same spirit that +the core crate is, just at a different layer (operations/RPC vs. +transport/auth/config). + +The HTTP crate depending on the call crate is "HTTP uses the call protocol +types," not "HTTP depends on SSH." This is within the spirit of this +ADR's decomposition. The call-crate → HTTP-crate edge is recorded +in the alkhttp crate overview (`overview.md`) and in the adapter +location map (see the alkcall crate docs, client-and-adapters). + +### Amendment 2 (2026-07-07): alknet-tty does not depend on the call crate + +Amendment 1's protocol-foundation framing was extended to alknet-tty in +an earlier draft ("alknet-tty depends on the call crate for the +`FrameFramedReader`/`FrameFramedWriter` framing utility"). A +pre-implementation sanity check found this was unsound: +`FrameFramedReader::read_frame()` is hardcoded to deserialize +`EventEnvelope` — the length-prefix read and the type-specific +deserialize are one entangled call, not a separable "framing utility." +alknet-tty's negotiation frame is a `NegotiateRequest`, not an +`EventEnvelope`, so `read_frame()` cannot return what alknet-tty needs; +the claimed reuse did not exist in a usable form. + +**Clarification:** alknet-tty does **not** depend on the call crate. +alknet-tty implements its own length-prefixed framing (~30 lines: 4-byte +big-endian length + UTF-8 JSON body) directly on tokio's +`AsyncRead`/`AsyncWrite`. The format coincides with the call crate's +framing by convention (both are length-prefixed JSON); the +implementations are independent. The Amendment 1 protocol-foundation +exception remains for the HTTP/agent/napi consumers (which use the call +crate's `OperationSpec`/`Handler`/`OperationAdapter` types — actual type +reuse, not framing glue); it no longer covers alknet-tty. See +ADR-057 (alknet-tty-no-alknet-call-dep, in the alknet mono-repo ADRs) +for the full decision +and the three options considered (duplicate / promote to core / use +the call crate). + +### Amendment 3 (2026-07-09): irpc is not a dependency of any crate + +The Decision table listed `irpc` as a dependency of the core crate +("tokio, quinn, rustls, irpc, iroh") and the call crate +("core, irpc"). This was carried over from the previous architecture +and never verified against the implementation: **no `.rs` file in the +workspace ever imported irpc**. The call protocol's wire format +(the call protocol's `protocol/wire.rs` in the call crate) is +hand-rolled length-prefixed JSON; the `EventEnvelope` shape was derived +from the `@alkdev/pubsub` TypeScript prior art (ADR-013), not from irpc. +The dead `irpc` / `irpc-derive` workspace deps and the call-crate consumer +dep were removed in commit `668d777`. See +ADR-064 (irpc-never-integrated-hand-rolled-framing, in the alknet +mono-repo ADRs) for the full +record (ADR-005, which accepted "irpc as the call protocol foundation," is +superseded). + +## Port notes + +- This is the historical alknet decomposition ADR, ported because alkhttp's + dependency edges are defined here (Amendment 1 in particular). The table's + crate names are annotated in place where the extraction renamed them: + alknet-core + alknet-call merged into **alkcall** (which vendors the core + types); alknet-vault → **alkvault**; alknet-http → **alkhttp**. The + dependency-flow diagram keeps the generic "core"/"call" labels it + historically used; "call ← alknet-agent / call ← alknet-napi" describe + the alknet mono-repo, not alkhttp's own edges. +- Per the task instructions, both amendments are retained. Amendment 1's + statement of the alkhttp edge ("alknet-http depends on alknet-call") + now reads as **alkhttp depends on alkcall alone** — the "depends on + `alknet-core, axum`" dependency in the table is now `alkcall, axum`. +- Link targets `crates/http/overview.md` and + `crates/call/client-and-adapters.md` (mono-repo doc paths) replaced: + the former points to `overview.md` in `docs/architecture/`, the latter + is a textual "alkcall crate docs" reference. ADR-057/ADR-064 links are + textual (those ADRs are not ported to alkhttp). +- ADR-057 and ADR-064, and the code-path references + (`crates/alknet-call/src/protocol/wire.rs`), are alkcall-internal + concerns; referenced textually. \ No newline at end of file diff --git a/docs/architecture/decisions/004-auth-as-shared-core.md b/docs/architecture/decisions/004-auth-as-shared-core.md new file mode 100644 index 0000000..95a200c --- /dev/null +++ b/docs/architecture/decisions/004-auth-as-shared-core.md @@ -0,0 +1,91 @@ +# ADR-004: Auth as Shared Core (IdentityProvider) + +*Ported from alknet ADR-004 (Auth as Shared Core (IdentityProvider)); re-targeted to alkhttp.* + +## Status + +Accepted + +## Context + +The previous architecture had authentication spread across multiple layers: `CredentialProvider` with four phases (A–D), `AuthProtocol` as an irpc service, `server_auth` and `client_auth` as separate modules, and `IdentityProvider` as a trait in the core. Different interface types presented credentials differently — SSH used key fingerprints, HTTP used Bearer tokens, DNS used query labels — but the resolution was ad-hoc and tied to the three-layer model. + +The ALPN dispatch model simplifies this: every handler receives the same `AuthContext`, but the credential extraction (how a handler learns who the peer is) differs per ALPN. The resolution (turning a credential into an `Identity`) should be shared across all handlers. + +## Decision + +> **Note**: The original text of this decision described the handler +> "enriching or replacing" the `AuthContext`. This was superseded by +> ADR-011, which made `AuthContext` immutable in `handle()` (passed as +> `&AuthContext`). Handlers resolve identity into a local variable and +> store it on `Connection` via `set_identity()`. The text below has been +> updated to reflect the ADR-011 model. + +Authentication and identity resolution live in the shared core (now vendored in alkcall) as shared infrastructure. Each handler presents credentials differently, but all resolve through the same `IdentityProvider`: + +```rust +pub trait IdentityProvider: Send + Sync + 'static { + fn resolve_from_fingerprint(&self, fingerprint: &str) -> Option; + fn resolve_from_token(&self, token: &AuthToken) -> Option; +} +``` + +Credential presentation per handler: + +| Handler | Credential presentation | Resolves via | +|---------|------------------------|-------------| +| SshAdapter | SSH public key handshake | `resolve_from_fingerprint()` | +| CallAdapter | AuthToken in first frame | `resolve_from_token()` | +| HttpAdapter | `Authorization: Bearer` header | `resolve_from_token()` | +| DnsAdapter | AuthToken in query labels | `resolve_from_token()` | +| WebTransportAdapter | AuthToken in CONNECT headers | `resolve_from_token()` | +| GitAdapter | Signed push certificate | `resolve_from_fingerprint()` | + +Auth resolution is **hybrid** — the endpoint resolves what it can, and handlers resolve what they must: + +1. **Endpoint-level resolution** (before `handle()` is called): If the TLS handshake provides a client certificate, the endpoint resolves the fingerprint to an `Identity` and passes it in `AuthContext`. This is the case for SSH (where the key exchange happens at the protocol level, but the TLS layer may also provide information). + +2. **Handler-level resolution** (inside `handle()`): For protocols that carry credentials in application frames (AuthToken in the first call frame, Bearer header in HTTP), the handler extracts the credential from the stream and calls `IdentityProvider` to resolve it. The handler then resolves the `Identity` into a local variable and stores it on the `Connection` via `set_identity()` for observability — it does **not** mutate the `AuthContext` (which is passed as `&AuthContext`, an immutable reference — see ADR-011). The per-request identity (for ACL) is resolved separately by the `CallAdapter` at `call.requested` time. + +The `AuthContext` passed to `handle()` may be partial — containing only transport-level information if no TLS client certificate was provided. Handlers must not assume `AuthContext` contains a fully resolved `Identity`. Each handler knows its own credential extraction protocol and is responsible for completing authentication. + +The `CredentialProvider` concept from the previous architecture is simplified: there is no phase progression (A–D). The `IdentityProvider` has two resolution paths — fingerprint and token — and a `ConfigIdentityProvider` implementation that draws from static and dynamic config. + +alkvault stays standalone. It does not depend on the core crate or `IdentityProvider`. The vault provides derived keys on request; identity resolution is a separate concern. + +## Consequences + +**Positive:** +- Unified identity model — every handler resolves identities the same way through `IdentityProvider` +- Handlers own their credential extraction — SSH reads key fingerprints, call reads AuthTokens, HTTP reads Bearer headers +- Endpoint provides what it can for free (TLS-level auth), handlers complete what they need +- Adding a new credential type is adding a method to `IdentityProvider`, not a new phase +- The vault stays standalone — no coupling between key derivation and identity resolution +- `AuthContext` is a value type — easy to construct in tests, can be partial for handler-level testing + +**Negative:** +- `IdentityProvider` is in the core crate — any change to it recompiles all handlers (mitigated: the trait should be stable; implementation changes don't force recompiles) +- Two resolution paths (fingerprint, token) may not cover all future auth schemes (mitigated: the trait can be extended, or a handler can do custom resolution after the initial AuthContext) +- Handlers must handle partial AuthContext — the endpoint may not have resolved an Identity, so handlers must be prepared to do credential extraction themselves +- WebTransport and browser-based auth needs careful design — AuthToken in CONNECT headers requires the token to be available before the stream is established + +## References + +- Pivot proposal (alknet mono-repo): `docs/research/pivot/alpn-service-architecture.md` +- [ADR-002](002-protocol-handler-trait.md): ProtocolHandler trait +- [ADR-003](003-crate-decomposition.md): Crate decomposition +- ADR-005: irpc as call protocol foundation +- The previous architecture had equivalent decisions in ADR-023 (unified auth) and ADR-029 (identity as core type), which are archived in the reference implementation at `/workspace/@alkdev/alknet-main/`. + +## Port notes + +- ADR-011 (AuthContext immutability) and the `IdentityProvider` internals are + alkcall-internal concerns; referenced textually without links. +- The `WebTransportAdapter` row and the WebTransport negative consequence are + retained as historical decision content; in alkhttp the browser path is + WebSocket (ADR-048) and h3/WebTransport is out of scope in alkhttp + (ADR-069, alknet-side). +- Crate renames: "alknet-core" → "the shared core (now vendored in alkcall)", + "alknet-vault"/"alknet-secret" → alkvault. (The source alternated between + the "alknet-secret" and "alknet-vault" names for the same crate in the + Consequences and Decision sections; normalized to alkvault.) \ No newline at end of file diff --git a/docs/architecture/decisions/010-alpn-router-and-endpoint.md b/docs/architecture/decisions/010-alpn-router-and-endpoint.md new file mode 100644 index 0000000..770bf32 --- /dev/null +++ b/docs/architecture/decisions/010-alpn-router-and-endpoint.md @@ -0,0 +1,328 @@ +# ADR-010: ALPN Router and Endpoint + +*Ported from alknet ADR-010 (ALPN Router and Endpoint); re-targeted to alkhttp.* + +## Status + +Accepted + +## Context + +ADR-001 establishes ALPN-based protocol dispatch: a single endpoint accepts connections, and the ALPN negotiated during the TLS handshake routes each connection to the correct `ProtocolHandler`. ADR-002 defines the `ProtocolHandler` trait. ADR-006 establishes one ALPN per connection. ADR-007 defines `Connection` and `BiStream`. + +The question is: **how does the endpoint work?** What accepts connections, negotiates ALPN, and hands connections to handlers? This is the central runtime piece of the shared core — every handler depends on it. The endpoint itself is an alkcall/alknet-side concern; this ADR is ported because alkhttp's `HttpAdapter` is a consumer of its dispatch model (registration, stealth mode, static ALPN registration), not because alkhttp owns an endpoint. + +### Multiple connectivity modes, not multiple transports + +The reference implementation supports three connectivity modes that serve **fundamentally different deployment contexts**: + +1. **QUIC+TLS (public)** — The node has a public IP and open ports. TLS provides protocol routing via ALPN negotiation. The TLS certificate is the node's **network-facing identity** — it's what clients verify when connecting to `alknet.example.com:4433`. This is the mode for replicators, VPS hosts, service providers. SSH key auth still handles **authentication** — the TLS cert is not the auth identity, it's the network identity. + +2. **iroh P2P (NAT traversal)** — The node has no public IP or open ports. iroh's relay handles NAT traversal and connection brokering. Node identity comes from iroh's `NodeId` (Ed25519 key pair). The relay is a signaling service, not a proxy — it helps peers establish direct QUIC connections. This is the mode for home servers, IoT devices, anything behind NAT. + +3. **TCP (local/dev)** — Bare SSH over TCP. Port 22. No TLS, no ALPN, no certs. SSH key exchange handles both identity and authentication. This is the mode for local network access and development. + +These are not interchangeable "transports" to be abstracted behind a trait. They are **different ways a node can be reached**, each with different identity and authentication implications: + +| Mode | Identity source | Auth mechanism | Requires public IP | Use case | +|------|-----------------|----------------|-------------------|----------| +| QUIC+TLS | TLS cert (network) + SSH key (auth) | SSH key, API key | Yes | VPS, replicators | +| iroh P2P | NodeId (Ed25519) | NodeId, SSH key | No | Home servers, NAT | +| TCP | SSH host key | SSH key | Yes (local) | Dev, LAN | + +### What the old "stealth mode" actually was + +The reference implementation's "stealth mode" is **SSH-over-TLS on port 443**. The TLS cert is NOT the node's identity — it's **camouflage**. The purpose is to make port 443 look like a web server to port scanners and DPI systems. Non-SSH traffic gets a fake nginx 404. SSH auth still happens via SSH key exchange *inside* the TLS tunnel. + +In the ALPN model, this concept maps to: the endpoint speaks TLS with ALPN, and the HTTP handler can serve a decoy website on `h2`/`http/1.1` while real services use `alk/ssh`, `alk/call`, etc. (ALPN prefix per the alkcall crate's ADR-004). The ALPN router does the "stealth" job — unknown ALPNs get the HTTP handler, which can serve whatever fronting content is desired. No byte-peeking needed. + +### iroh produces QUIC connections with ALPN + +iroh's `Endpoint::accept()` produces incoming QUIC connections with ALPN negotiation (step 4 of iroh's connection establishment). The `iroh::Endpoint` supports `set_alpns()` to configure which ALPNs the endpoint advertises — the same mechanism iroh's own `Router` uses internally. + +This means the iroh integration is not a separate dispatch path. It uses the **same ALPN dispatch** as the quinn path. The `iroh::Endpoint` accepts connections, negotiates ALPN, and our `HandlerRegistry` dispatches to the right handler — exactly like iroh's own `Router` does with its `ProtocolMap`. + +We do NOT wrap iroh's `Router`. We use `iroh::Endpoint` directly and run our own accept loop, because: +- Our `HandlerRegistry` is shared between quinn and iroh connection sources +- Our `AuthContext` construction differs per connection source +- Our shutdown and error handling patterns are our own + +The relationship is: **iroh's Router is a reference implementation of the pattern we're building.** Our endpoint generalizes it to support multiple connection sources with the same dispatch. + +### Key design questions + +1. **How many endpoints can a node have?** A node may need to listen on quinn (public QUIC+TLS) AND iroh (P2P relay) simultaneously. These are not alternatives — they're complementary connectivity modes. +2. **Handler registration**: Static (at startup) or dynamic (at runtime)? +3. **Connection lifecycle**: Who owns the endpoints? How does graceful shutdown work? +4. **Error handling**: What happens when a handler panics? When ALPN negotiation fails? + +## Decision + +### A node can have multiple endpoints + +The endpoint type manages one or more QUIC connection sources. Each source produces connections that feed into the same `HandlerRegistry`: + +```rust +pub struct Endpoint { + // One or more QUIC connection sources + quinn: Option, // Public QUIC+TLS + iroh: Option, // P2P relay-assisted + + handlers: Arc, + dynamic: Arc>, + identity_provider: Arc, + shutdown: watch::Receiver, +} +``` + +A node that has a public IP runs with `quinn: Some(...)` — it listens on a public address with TLS+ALPN. A node behind NAT runs with `iroh: Some(...)` — it connects to a relay and accepts P2P connections. A node that has both runs with both — it's reachable via either path, and both feed into the same ALPN router. + +**TCP mode is not an endpoint concern.** TCP mode in the reference implementation is SSH over raw TCP on port 22. This is not QUIC and doesn't have ALPN. In the new model, TCP access to SSH is handled by the SSH handler directly — it can listen on a TCP socket independently of the ALPN endpoint. This is a handler-specific concern, not a core endpoint concern. + +### HandlerRegistry maps ALPN strings to ProtocolHandler instances + +```rust +pub struct HandlerRegistry { + handlers: HashMap<&'static [u8], Arc>, +} +``` + +Registration is static at startup (OQ-04). The CLI binary constructs a `HandlerRegistry`, inserts handlers, and passes it to `Endpoint::new()`. + +The ALPN strings for the quinn endpoint's TLS `ServerConfig` are derived from the registry's keys. The iroh endpoint's ALPN strings are also derived from the registry — both endpoints advertise the same set of ALPNs. + +### Accept loop: accept from all sources, dispatch by ALPN + +The endpoint runs accept loops for each active connection source. All loops dispatch through the same `HandlerRegistry`: + +``` +// Quinn accept loop (if configured) +loop { + incoming = quinn_endpoint.accept().await + connection = incoming.await // TLS handshake + ALPN negotiation + dispatch(connection) +} + +// iroh accept loop (if configured) +loop { + incoming = iroh_endpoint.accept().await + connection = incoming.await // iroh QUIC connection + ALPN + dispatch(connection) +} + +fn dispatch(connection) { + alpn = connection.alpn() + handler = registry.get(alpn) + match handler { + Some(h) => { + auth = AuthContext::from_connection(&connection) + conn = Connection::from_quinn(connection) // or from_iroh + tokio::spawn(h.handle(conn, &auth)) + } + None => connection.close() + } +} +``` + +Both accept loops are `tokio::select!`-ed against the shutdown signal. + +### TLS certificate and the distinction between network identity and auth identity + +For the quinn endpoint, the TLS cert serves as **network-facing identity** — it's what clients verify when connecting to a domain name. It is NOT the node's authentication identity. Authentication is handled by handlers (SSH key exchange, API tokens, etc.). + +This is the same model as the reference implementation's TLS mode: the cert makes the port look legitimate and encrypts traffic, but SSH key exchange handles the actual authentication. The ALPN model extends this: the cert + ALPN routing is the network layer, handler-specific auth is the application layer. + +For the iroh endpoint, the `NodeId` serves as network identity. No TLS cert is needed — iroh's QUIC uses the NodeId for connection verification. + +### RFC 7250: Raw Public Keys in TLS + +iroh uses RFC 7250 raw public keys instead of X.509 certificates for TLS. The implementation is strikingly simple (see `iroh/iroh/src/tls/resolver.rs`): take an Ed25519 key, wrap its SPKI public key as a `CertificateDer`, and tell rustls `only_raw_public_keys() -> true`. No X.509, no CAs, no domain names, no cert renewal. + +rustls already supports RFC 7250. This means the quinn endpoint can also use raw Ed25519 public keys instead of X.509 certs. The implications: + +1. **No domain required.** A node without a domain name can use raw public keys for the quinn path — the same key-based identity model as iroh, but with direct QUIC over UDP instead of relay-assisted connections. +2. **Key = identity.** The Ed25519 public key IS the node's identity. No CA trust chain, no cert expiry, no renewal. The key is derived from alkvault or generated at startup. +3. **X.509 is optional.** Domain-facing identity (for replicators, public services) uses X.509 certs. Key-based identity (for personal nodes, P2P) uses raw public keys. Both work with the same quinn endpoint. +4. **Browser compatibility.** Browsers don't support RFC 7250 — they require X.509. For browser/WebTransport clients, X.509 certs are needed. For alknet-native clients, raw public keys work fine. + +This reframes the connectivity model. The quinn and iroh paths are not distinguished by their identity model (both can use Ed25519 keys). They're distinguished by how the connection is established: + +| Path | Connection establishment | Identity model (v1) | Identity model (future) | +|------|------------------------|--------------------|-------------------------| +| quinn | Direct UDP, public IP | X.509 (domain) | X.509 or RFC 7250 raw key | +| iroh | Relay-assisted P2P | RFC 7250 raw key (NodeId) | Same | + +### Error taxonomy + +> **`EndpointError` is removed** per ADR-083 (Amendment 2026-07-15 + +> the `EndpointError`-removal amendment; see the alkcall crate docs). +> `BindFailed` is vestigial (the endpoint takes pre-bound transports); +> `TlsConfig` is removed (the endpoint takes no TLS config); +> `HandlerNotFound` is swallowed by `dispatch` (close + log, not an +> error). `shutdown()` is infallible. The sketch below is the historical +> shape; it does not survive into the endpoint crate. `HandlerError` is +> unchanged. + +```rust +// HISTORICAL — removed per ADR-083. See the note above. +pub enum EndpointError { + BindFailed(io::Error), + TlsConfig(io::Error), + HandlerNotFound(Vec), // ALPN string with no registered handler +} + +pub enum HandlerError { + ConnectionClosed, + StreamError(io::Error), + AuthRequired, + Internal(Box), +} +``` + +- ~~`EndpointError`~~: **removed** (ADR-083; see the alkcall crate docs). The endpoint takes pre-built transports and swallows no-handler matches; `shutdown()` is infallible. +- `HandlerError`: Problems within a handler's `handle()` method. Non-fatal — the connection is closed, but the endpoint keeps running. + +## Consequences + +**Positive:** +- A node can be reachable via multiple paths simultaneously (public QUIC+TLS, iroh P2P) +- ALPN router is transport-agnostic — dispatches by ALPN string regardless of connection source +- Adding a handler is registering an ALPN string — no endpoint code changes +- Handler panics are isolated — one bad handler can't take down the endpoint +- "Stealth mode" maps naturally to the HTTP handler serving decoy content on `h2`/`http/1.1` — in alkhttp this is the `HttpAdapter`'s decoy surface +- Both iroh and quinn produce QUIC connections — same `Connection` type works for both + +**Negative:** +- The core crate depends on both quinn and iroh (mitigated: both are feature-gated; a node that only needs one doesn't compile the other) +- The endpoint is more complex than a single quinn listener — it manages multiple accept loops +- TLS identity provisioning has two distinct use cases: RFC 7250 raw keys (default for P2P/key-based identity) and X.509 certs (for domain-hosted services and browsers). ACME auto-provisioning and RawKey decoupling from the `iroh` feature are designed in ADR-027 (see the alkcall crate docs). See OQ-12. +- No runtime handler registration without regenerating the TLS config (mitigated: two-way door, start static, add ArcSwap later if needed) + +## References + +- [ADR-001](001-alpn-protocol-dispatch.md): ALPN-based protocol dispatch +- [ADR-002](002-protocol-handler-trait.md): ProtocolHandler trait +- ADR-006: ALPN string convention and connection model (see the alkcall crate docs) +- ADR-007: BiStream type definition — Connection, SendStream, RecvStream (see the alkcall crate docs) +- ADR-009: One-way door decision framework (alknet mono-repo) +- OQ-04: Dynamic handler registration (two-way door, start static) +- OQ-05: Multi-transport endpoint (now: multi-connectivity endpoint) +- iroh Router pattern (alknet mono-repo): `docs/research/references/iroh/` +- Reference implementation (alknet mono-repo): `alknet-main/crates/alknet-core/src/server/serve.rs` +- Reference stealth mode (alknet mono-repo): `alknet-main/crates/alknet-core/src/server/stealth.rs` +- Reference iroh transport (alknet mono-repo): `alknet-main/crates/alknet-core/src/transport/iroh_transport.rs` + +## Amendments + +### Amendment 1 (2026-07-09): TCP+TLS can dispatch through the ALPN router via `from_stream` + +This ADR's Decision section states: **"TCP mode is not an endpoint concern."** +The rationale was that bare TCP (SSH over port 22) does not use QUIC or +ALPN, so TCP access is handled by individual handlers listening on a TCP +socket independently — a handler-specific concern, not a core endpoint +concern. + +That rationale holds for *bare TCP* (no TLS, no ALPN). But ADR-065 +(see the alkcall crate docs) adds +`Connection::from_stream` / `from_bidi`, which construct a `Connection` +from any `AsyncRead + AsyncWrite` pair — including a +`TlsStream`. A TCP+TLS accept loop can now call +`Connection::from_bidi(tls_stream, alpn, remote_addr)` and dispatch through +the **same `HandlerRegistry`** as QUIC connections, by the ALPN negotiated +in the TLS handshake. This is not a parallel listener bypassing the core — +it's the same ALPN dispatch, over a non-QUIC transport. + +**Revised reading of "TCP is not an endpoint concern":** the +endpoint struct (quinn + iroh) remains QUIC-only — the endpoint +does not own a TCP+TLS accept loop. But a TCP+TLS accept loop can be +constructed *outside* the endpoint (by the assembly layer or a handler) +and feed connections into the same `HandlerRegistry` the endpoint uses. +The endpoint is one accept-loop source; a TCP+TLS loop is another source +that shares the registry. The "not an endpoint concern" framing is +preserved at the struct level (no `tcp: Option` on +the endpoint); the "TCP can't participate in ALPN dispatch" framing +is **reversed** — `from_stream` is the primitive that lets TCP+TLS +participate without changing the endpoint design. + +The unblocked follow-ups (not part of ADR-065, but enabled by it): + +- **Standard HTTP over TCP+TLS** (`api.alk.dev`'s requirement): a TLS + accept loop wraps each `TlsStream` as a `Connection` via + `from_bidi` and dispatches to `HttpAdapter` by the negotiated ALPN + (`h2`/`http/1.1`). `HttpAdapter::handle` runs hyper over a + bidirectional stream (BiStream) yielded by `Connection::accept_bi()` — + unchanged from the QUIC path. No handler code changes. +- **SSH channel dispatch**: an SSH handler wraps each russh channel as a + `Connection` via `from_stream` and dispatches by channel-type (treated as + the ALPN string) through `HandlerRegistry`. One SSH connection carries + heterogeneous channels — a multiplexing power QUIC's per-connection ALPN + doesn't provide natively. +- **WebTransport stream dispatch** (parked per + [ADR-044](044-defer-webtransport-browsers-use-websocket.md), + unblocked structurally; an alknet-side concern — out of scope in + alkhttp, ADR-069): the WT handler wraps each WT stream via + `from_stream`. + +The `iroh 0.35 → 1.0.2` migration (commit `acd049e`, 2026-07-09) is a +related cleanup: it bumps the iroh dep to 1.0, unblocking +`alknet-blobs` (which pulls `iroh 1.0` transitively). It is not an +architectural change — 6 API surface edits in `endpoint.rs` / +`types.rs` (the `Endpoint::builder` preset, +`SecretKey::from_bytes`/`generate` signatures, +`Connection::remote_id`/`alpn` return types). No ADR needed; the +endpoint design is unchanged. + +### Amendment 2 (2026-07-14): TCP+TLS is a first-class owned transport (supersedes Amendment 1's struct-level exclusion) + +Amendment 1 preserved the "not an endpoint struct concern" framing at +the struct level — no `tcp: Option` field on the +endpoint. The rationale was that the endpoint built transports +internally (quinn, iroh), and TCP+TLS couldn't fit that construction +shape, so it was a sibling loop outside the struct. + +ADR-083 (see the alkcall crate docs) removes that rationale: the +endpoint no longer builds transports at all — it runs accept loops on +whatever it's given via builder methods. TCP+TLS is a listener +transport, same shape as quinn and iroh (accept → extract ALPN + +fingerprint → `Connection::from_bidi` → `dispatch`). The endpoint now +owns it via `with_tcp_tls(listener, acceptor)` (behind a `tcp` +feature), runs its accept loop inside `run()`, and stops it on +`shutdown()`. The struct gains a `tcp_tls: Option` +field. + +Amendment 1's *dispatch* contribution survives — the public `dispatch` +method and `Connection::from_bidi` are what make TCP+TLS dispatch work. +Amendment 1's *struct-level exclusion* (no `tcp` field, sibling loop +outside) is **superseded**: TCP+TLS is now a first-class owned transport. +The `dispatch` method stays public, but for genuinely external shapes +(SSH channels, future WebTransport streams) — connection-internal +multiplexing, not listener transports. + +This also means shutdown is single-owner: the endpoint owns all its +accept loops (quinn, iroh, TCP+TLS); one `shutdown()` stops them all. +The multi-owner shutdown coordination problem (OQ-61) does not arise. + +## Port notes + +- This ADR is endpoint-internal to alkcall/alknet; ported because alkhttp + consumes its dispatch model (HttpAdapter registration, stealth/decoy + mapping, static ALPN registration). The alknet `AlknetEndpoint` struct in + the sketches is generically renamed `Endpoint` — the type lives in + alkcall's lineage, not in alkhttp. +- Stealth-mode phrasing retargeted: "the `alknet/http` handler can serve a + decoy website" → "the HTTP handler can serve a decoy website", with a + consequence clause noting that in alkhttp this is the `HttpAdapter`'s + decoy surface. +- QUIC-overload phrasing made transport-agnostic in the two places that + describe HttpAdapter mechanics: Amendment 1's "calls `accept_bi` once + (yielded by the single stream)" now reads "runs hyper over a + bidirectional stream (BiStream) yielded by `Connection::accept_bi()`". + The endpoint/iroh/quinn mechanics sections are untouched (they describe + alkcall/alknet-side reality). +- ADR-065, ADR-083, ADR-027, ADR-044 references: ADR-044 is linked as + `decisions/044-defer-webtransport-browsers-use-websocket.md`-equivalent + (ported by other agents under the same number); ADR-065, ADR-083, and + ADR-027 are alkcall-internal, so referenced textually. h3/WebTransport + mentions that imply an alkhttp deliverable are marked out of scope in + alkhttp (ADR-069). +- Mono-repo reference paths annotated as alknet mono-repo paths. \ No newline at end of file diff --git a/docs/architecture/decisions/014-secret-material-flow-and-capability-injection.md b/docs/architecture/decisions/014-secret-material-flow-and-capability-injection.md new file mode 100644 index 0000000..4deb2b2 --- /dev/null +++ b/docs/architecture/decisions/014-secret-material-flow-and-capability-injection.md @@ -0,0 +1,240 @@ +# ADR-014: Secret Material Flow and Capability Injection + +*Ported from alknet ADR-014 (Secret Material Flow and Capability Injection); re-targeted to alkhttp.* + +## Status + +Accepted + +## Context + +alkvault holds the master seed and can derive keys and encrypt/decrypt +arbitrary data. ADR-008 established that the vault is a **capability source**: +"derived keys and decrypted credentials are injected into operation contexts +at the assembly layer, not passed as vault references to handlers." That +prose was correct but the mechanism was never specified. + +The result was a contradiction in the spec documents. ADR-008 said the master +seed never crosses the network, but `operation-registry.md` showed +`vault/derive`, `vault/unlock`, and `vault/decrypt` registered as call protocol +operations — directly on the wire. Those two statements cannot both be true. +The contradiction arose because no injection mechanism existed in the +architecture, so the only way the docs could show a handler obtaining a key was +to expose vault operations over the call protocol. + +This is a one-way door. Once secret material crosses the wire as a call +protocol operation, the attack surface is permanent: + +- `vault/unlock` accepts a BIP39 mnemonic — the root of trust — over the wire. + A compromised peer, a logging accident, a tracing span, and the seed is gone. +- `vault/derive` returns a `DerivedKey`. The type redacts the private key in + JSON today, but the operation's existence means a serialization change, a + binary codec addition, or a wrapper change would leak it. The surface is + the risk, not the current implementation. +- `vault/decrypt` accepts an encrypted blob and returns plaintext. Any + authorized caller can decrypt any blob they possess. + +The broader problem this decision addresses is structural: the industry +default for storing LLM provider keys, API tokens, and other credentials is +plaintext config files and environment variables (e.g., the aisdk Rust port +reads `std::env::var("GOOGLE_API_KEY")` and the example backend calls +`dotenv::dotenv()`). The alk stack replaces that with a vault (alkvault). But +the vault only solves the storage problem; the flow problem — how decrypted +material reaches the code that needs it without crossing the network — +requires its own decision. + +There is a separate, second axis that the current `OperationContext` conflates +with the secret-flow problem. A handler has two orthogonal credential concerns: + +- **Identity (inbound)**: who is calling me? Resolved per-request from + `AuthContext` (TLS client cert, auth token). Already in `OperationContext`. +- **Capabilities (outbound)**: what secrets can I use for outbound calls? This + is the missing axis. A handler calling Google's API needs a decrypted Google + API key. That is not the caller's identity — it is the handler's own outbound + credential, provisioned by the assembly layer. + +Mixing these two into one channel (e.g., stuffing secrets into +`OperationContext.metadata: HashMap`) is a leak risk: metadata +propagates through nested calls via `OperationEnv::invoke()`, so a secret +placed there by one handler would flow to every downstream operation. + +## Decision + +**1. The vault is assembly-layer only.** + +The CLI binary (or an embedded assembly layer) is the sole component that +talks to `VaultServiceHandle` directly. It unlocks the vault at startup, +derives and decrypts what each handler needs, and constructs handlers with the +results. No vault operation (`derive`, `decrypt`, `unlock`, `lock`) is +registered as a call protocol operation. The vault has no ALPN. The master +seed and derived private keys never enter the call protocol. + +**2. Capabilities are the injection mechanism.** + +A `Capabilities` type carries outbound secret material from the assembly layer +into handlers. Capabilities are distinct from identity (inbound auth) and +distinct from per-request metadata. The concrete shape of the `Capabilities` +type is a two-way door — to be decided during implementation of the call +protocol crate (now the alkcall crate). The one-way constraint is: + +- Capabilities hold non-serializable, zeroized secret material. They cannot + cross the call protocol wire even by accident — they are not + `serde_json::Value`, they do not implement `Serialize`, and they do not + appear in `EventEnvelope` payloads. +- Capabilities are injected at handler construction (the common case: a static + decrypted API key held for the handler's lifetime) or scoped per-request for + internal-only flows. They are never populated from call protocol + inputs. + +**3. The call protocol carries no secret material.** + +This is a wire-level constraint on the call protocol, not a handler-level +convention. Secret material (private keys, API keys, mnemonics, decrypted +credentials, raw tokens) must not appear in: + +- `call.requested` payloads (inputs) +- `call.responded` payloads (outputs) +- `OperationContext.metadata` + +The wire format does not enforce this — it carries `serde_json::Value` — so the +constraint is architectural, enforced by the operation registry and by +convention. Operations that need to share public key material (e.g., for +identity verification) use a dedicated operation that returns only the public +component, never the private key. + +**4. Adapters take credential sources, not static tokens.** + +The `from_openapi` and `from_jsonschema` adapter patterns (the +`OperationAdapter` trait lives in the alkcall crate per alkcall ADR-033; the +HTTP-backed adapter implementations live in alkhttp — see ADR-066) register +HTTP-backed operations. The TypeScript `@alkdev/operations` `from_openapi` +takes `config.auth: { token: "..." }` — a static string. The Rust adapters +take a credential source wired to the assembly layer (a resolver, a capability +handle, or an injected secret), not a literal token. This is the integration +point where the vault feeds credentials into HTTP-backed operations: the +assembly layer decrypts the token at startup and provides it to the adapter at +registration time. + +**5. Handlers that need per-request vault access receive a scoped capability.** + +The common case (a static decrypted API key) is covered by construction-time +injection. A narrower case — a handler that derives a child key for a specific +operation (e.g., signing for GitHub authentication) — receives a +scoped capability that can only derive at a restricted path set. This is still +not a vault reference: it is a restricted handle that performs a specific +derivation and returns the result to the handler, in-process. The handler +never sees the master seed. Whether this scoped capability is a distinct type +or modeled as a pre-derived key injected at construction is a two-way door +left to the alkcall and agent crate specs. + +## Consequences + +**Positive:** + +- The master seed and derived private keys never cross the network. The attack + surface for the root of trust is local-only. +- The `OperationContext` gains a clean second axis (capabilities) instead of + overloading `metadata` for secrets, preventing accidental propagation of + secret material through nested calls. +- Handlers that need outbound credentials (the agent handler calling an LLM + provider) receive them directly — no indirection through a `vault/derive` + call, no latency, no failure mode where the vault must be reachable at call + time. +- The adapter contract (alknet OQ-15) gains a concrete shape: adapters take a + credential source from the assembly layer, not a static token. This makes + the `from_openapi` / `from_jsonschema` / `from_call` patterns safe by + construction. +- The model is structurally incompatible with the env-var / plaintext-config + default. There is no `std::env::var("API_KEY")` path — the only way a handler + gets a credential is through a capability, and the only way a capability is + populated is through the assembly layer from the vault. + +**Negative:** + +- The assembly layer (CLI binary) has more construction-time responsibility: it + must know which handlers need which credentials and wire them. This is + expected — the CLI assembles everything (alknet ADR-008). +- Adding a new handler that needs a new credential requires updating the + assembly layer, not just registering an operation. This is a feature, not a + bug: it forces an explicit decision about what secret material a handler + needs. +- Remote vault administration (unlock a running node's vault over the network) + is not supported by this decision. If that capability is needed in the + future, it would require a separate, heavily restricted mechanism (admin + scope, mTLS-only, never expose the mnemonic over an unauthenticated channel) + and its own ADR. This decision does not close that door; it simply does not + open it. +- The `Capabilities` type shape is not fully specified here. The one-way + constraint (non-serializable, zeroized, injection-only) is fixed; the + concrete API is a two-way door for the call protocol spec. + +## Assumptions + +These are the load-bearing assumptions. If any of them breaks, the decision +should be revisited: + +1. **Handlers need credentials at construction time or at call time, not + dynamically discovered at call time.** If a handler needs to derive a key + at an unpredictable path determined by call input, the scoped-capability + model still covers it (the handler holds a scoped vault access), but the + surface area is larger. The assumption is that this case is rare. +2. **The call protocol's threat model excludes the assembly layer.** The CLI + binary is trusted to hold the vault handle and inject capabilities. If the + assembly layer is compromised, all handlers' capabilities are compromised. + This is the same trust boundary as alknet ADR-008. +3. **No legitimate use case requires returning a private key over the wire.** + Public key sharing (identity verification, encryption to a recipient) is + the only cross-node key material flow. If a use case for returning a + private key emerges (e.g., a key-escrow service), it needs its own ADR and a + very different threat model. +4. **Adapters are registered at startup, not at call time.** The credential + source is wired to the adapter when the operation is registered, not when + the operation is invoked. This is consistent with alknet OQ-04 (static + registration at startup). + +## References + +- alknet ADR-003: Crate decomposition (alknet-vault was standalone; now + extracted as alkvault) — [ported here](003-crate-decomposition.md) +- alkcall ADR-013: irpc as call protocol foundation (alknet ADR-005; + superseded — see alkcall ADR-014, irpc was never integrated) +- alknet ADR-008: Vault integration point (capability source — this ADR + specifies the mechanism that ADR-008 described in prose; alknet mono-repo + ADR — the vault is an alkvault concern) +- alknet ADR-009: One-way door decision framework (now alkcall ADR-032) +- alkcall ADR-033: Rust as canonical implementation language (alknet ADR-013) +- alknet OQ-15: Call protocol client and adapter contract (this ADR constrains + the adapter contract: adapters take credential sources, not static tokens; + resolved by alkcall ADR-022, ported here as + [ADR-017](017-call-protocol-client-and-adapter-contract.md)) +- alknet OQ-16: Safe vault operations for call protocol exposure (resolved by + this ADR: none, for now) +- alkvault crate (extracted from the alknet mono-repo's `alknet-vault`) + +## Port notes + +- Renames: "alknet-http" → alkhttp; "alknet-core"/"alknet-call" → alkcall + (alkcall 0.1.1, which merged the old alknet-core and alknet-call and vendors + the core types); "alknet-vault" → alkvault; "the `alknet` crate" (CLI + binary) → "the CLI binary (or an embedded assembly layer)" — the CLI crate + name is a consumer concern, not an alkhttp one. +- "over QUIC" → "over the wire" (§Context, `vault/unlock` bullet): the call + protocol is transport-agnostic (alkcall ADR-007), so the constraint is + transport-independent. +- §4 correction: the original said the adapter patterns are "defined in Rust + in alknet-call per ADR-013". The `OperationAdapter` trait lives in the + alkcall crate (alkcall ADR-033, alkcall ADR-022 §5); the HTTP-backed adapter + implementations live in alkhttp (alkcall ADR-027 / this crate's ADR-066). + The original predates ADR-066's location correction. +- References to the alknet mono-repo's vault ADRs (ADR-008) and OQs (OQ-04, + OQ-15, OQ-16) are annotated as alknet-record citations; the alkcall crate + has its own OQ numbering. OQ-15's resolution (the adapter contract) is + alkcall ADR-022, ported to this crate as ADR-017. +- alknet ADR-003 is ported to this crate under the same number and is linked; + alknet ADR-005/008/009/013 are cited textually with their alkcall mappings + where one exists (alkcall ADR-032 for the one-way-door framework, alkcall + ADR-033 for Rust-canonical). +- No decision content changed — the vault-is-assembly-layer-only rule, the + Capabilities injection mechanism, the no-secret-material-on-the-wire + constraint, and the credential-source adapter contract are verbatim from + the alknet ADR. \ No newline at end of file diff --git a/docs/architecture/decisions/015-privilege-model-and-authority-context.md b/docs/architecture/decisions/015-privilege-model-and-authority-context.md new file mode 100644 index 0000000..cf750cd --- /dev/null +++ b/docs/architecture/decisions/015-privilege-model-and-authority-context.md @@ -0,0 +1,340 @@ +# ADR-015: Privilege Model and Authority Context + +*Ported from alknet ADR-015 (Privilege Model and Authority Context); re-targeted to alkhttp.* + +## Status + +Accepted + +## Context + +The call protocol allows handlers to compose other operations through +`OperationEnv::invoke()`. This creates a call tree: a parent request spawns +children, which may spawn their own children. The `parent_request_id` field +records this tree. + +The previous design had a `trusted: bool` flag on `OperationContext`. When a +handler invoked another operation through `OperationEnv`, the nested call was +marked `trusted: true` and **all ACL checks were skipped**. The intent was to +avoid double-checking: if `/agent/chat` is allowed and it internally calls +`/auth/verify`, the auth check is "trusted" because the caller already passed +ACL on `/agent/chat`. + +This is a privilege escalation vector. Two concrete attacks: + +**Buggy handler**: a handler accidentally calls an operation it shouldn't. With +`trusted: true`, ACL is skipped entirely. A handler with `read` scope that +accidentally calls an operation requiring `admin` succeeds — the caller's `read` +scope effectively triggered an `admin` operation. + +**Parameterized dispatch**: a handler takes caller input that determines which +internal operation to call. This is the core agent use case — an LLM picks which +tool to invoke based on the user's prompt. With `trusted: true`, the LLM (and +therefore the user) can invoke any registered operation without ACL checks, +regardless of the caller's scopes. A caller with `chat` scope can invoke +operations requiring `admin` by choosing the right tool name. + +The call protocol is a general-purpose cross-boundary RPC mechanism. Every +consumer — NAPI adapter, Python adapter, agent service, HTTP adapter, future +services — inherits whatever privilege model the protocol defines. The +privilege boundary between external and internal calls, and the authority +context switch for composition, are core protocol semantics. This is not a +feature of any single consumer; it is the protocol's security model. + +The agent service is a useful test case because it exercises every edge case +(parameterized dispatch, deep composition, dynamic operations, role-based +escalation), but the decision belongs to the call protocol. It is recorded in +the alkcall crate (alkcall ADR-017) and ported to alkhttp because the HTTP +gateway and the adapters are consumers of this model: the gateway's +`AccessControl`-filtered `/search` discovery and the adapter-registered +`Internal`-by-default operations both sit on top of these protocol semantics. + +## Mental Models + +Two analogies clarify the model: + +**Kernel/user mode**: external operations are syscalls — curated entry points +where an unprivileged caller can enter the kernel. Internal operations are +kernel functions — callable only from composition, not from userspace. The +`internal` flag means "this call is in kernel mode." Kernel mode has access +controls — it runs under a different principal, not with no principal. + +**Domain/integration events**: external operations are integration events — +they cross a boundary and are visible to external systems. Internal operations +are domain events — they stay within the bounded context. `services/list` is +the integration contract; it only exposes integration events. + +**Principal/agent (legal contracting)**: the caller is the principal; the +handler is the agent. The principal delegates scoped authority to the agent. +The agent acts under its own identity (for attribution) but with the principal's +delegated authority (for scope). Liabilities flow upstream (traceable through +`parent_request_id`); privileges flow downstream (the agent gets a subset of the +principal's authority). Role-based escalation: a lower-privileged role can +escalate through a chain of command (agent requests promotion, architect +performs it), not through direct authority. + +## Decision + +### 1. The `internal` flag switches authority context, not skips ACL + +The `internal` flag on `OperationContext` marks calls that originated from +composition (a handler calling another operation via `OperationEnv`), as opposed +to external calls that arrived as `call.requested` from a wire caller. + +When `internal: true`: +- The ACL check runs against the **handler's identity** (set at registration by + the assembly layer), not the caller's identity and not as a blanket skip. +- The handler's identity has scopes scoped to its composition needs (least + privilege), not blanket root and not the caller's scopes. + +When `internal: false` (external call from the wire): +- The ACL check runs against the **caller's identity** (from `AuthContext`, + resolved per-request). + +The `internal` flag is set by `OperationEnv`, not by callers. A handler cannot +mark its own call as internal. The field uses module-private construction; only +`pub fn is_internal(&self) -> bool` is exposed for reads. + +### 2. Operations have External/Internal visibility + +`OperationSpec` has a `visibility: Visibility` field: + +```rust +pub enum Visibility { + External, // Callable from the wire (call.requested from a consumer) + Internal, // Composition-only (env.invoke from a handler) +} +``` + +The assembly layer declares visibility when registering operations. + +When a `call.requested` arrives from a wire consumer: +- An `Internal` operation returns `call.error` with code `NOT_FOUND` (not + `FORBIDDEN`). This does not leak that the operation exists. +- An `External` operation proceeds to ACL checking. + +`services/list` only returns `External` operations to remote callers. Internal +operations are not part of the wire-facing API surface. A remote consumer cannot +enumerate the internal call tree. + +### 3. Handler identity is carried on OperationContext + +> **Note**: This decision's `handler_identity: Option` type was +> superseded by alkcall ADR-018 (alknet ADR-022, ported here as +> [ADR-022](022-handler-registration-provenance-and-composition-authority.md)), +> which replaced `Identity` with `CompositionAuthority` — a declared authority +> bundle that is not a peer identity and is not resolvable through +> `IdentityProvider`. The core decision (authority switch, not ACL skip) holds +> unchanged. See ADR-022 Decision 2 for the current type. + +`OperationContext` carries both the caller's identity (who invoked me) and +the handler's identity (who am I acting as): + +```rust +pub struct OperationContext { + pub request_id: String, + pub parent_request_id: Option, + pub identity: Option, // Caller's identity (inbound) + // Type changed to Option by ADR-022: + pub handler_identity: Option, // Handler's composition authority + pub capabilities: Capabilities, + pub metadata: HashMap, + // env/scoped_env split by ADR-024 (now alkcall ADR-019): + pub scoped_env: ScopedOperationEnv, // Reachability data (ADR-022, ADR-024) + pub env: Arc, // Dispatch trait (ADR-024) + /// Module-private for writes; read via `is_internal()`. Set only by + /// `OperationEnv::invoke()` (true) or `CallAdapter` dispatch (false). + pub(crate) internal: bool, +} + +impl OperationContext { + pub fn is_internal(&self) -> bool { self.internal } +} +``` + +- `identity`: the authenticated caller (from `AuthContext`). For external calls, + this is who sent the `call.requested`. For internal calls, this is the + *parent handler's* identity (propagated through `OperationEnv::invoke()`). +- `handler_identity`: the identity of the handler processing this call. Set at + registration by the assembly layer. For external calls, this is the handler's + own identity. For internal calls, the ACL check runs against this identity. + +The distinction is the principal/agent model: `identity` is the principal (who +delegated), `handler_identity` is the agent (who is acting). Attribution traces +through both — any action can be attributed to the handler that performed it and +the caller that initiated the chain. + +### 4. Scoped composition env + +The `OperationEnv` given to a handler is scoped — it can only invoke a declared +set of operations. This bounds the parameterized-dispatch attack surface: a +caller (or an LLM) picking which operation to invoke picks from the declared +set, not from the entire registry. + +Scoping happens at two levels: + +**Static scoping at registration**: the assembly layer declares which operations +a handler may compose. The `OperationEnv` given to that handler is pre-filtered +— `invoke("fs", "readFile", ...)` works, `invoke("admin", "deleteUser", ...)` +returns `NOT_FOUND`. This is the reachability control. + +**Dynamic scoping at sandbox creation**: when a handler spawns a sandbox +(quickjs), it passes a *further scoped* env to the sandbox — a subset of what +the handler itself can reach. The handler might have `fs:read` and `bash:exec`, +but it only gives the sandbox `fs:read` (not `bash:exec`), because the sandbox +runs untrusted LLM-generated code. This is the "privileges flow downstream" +principle: the principal delegates a subset. + +The specific API for declaring the scoped operation set is specified in +[ADR-022](022-handler-registration-provenance-and-composition-authority.md) +(`ScopedOperationEnv { allowed_operations: HashSet }`), +operation-level granularity (not just namespace-level). This is finer-grained +than the TypeScript `@alkdev/operations` `buildEnv()` which used +`allowedNamespaces` — operation-level scoping is safer for the +parameterized-dispatch use case. + +### 5. The three controls together + +The three controls are independent and all are needed: + +| Control | What it gates | Without it | +|---------|--------------|-----------| +| Operation visibility | Whether an operation is callable from the wire | Internal operations exposed to external callers | +| Handler identity | What authority composition runs under | ACL skipped or caller's scopes propagated (escalation) | +| Scoped composition env | What operations a handler can reach | Handler can call anything in the registry | + +- Visibility alone: internal operations are hidden from the wire, but + composition skips ACL (escalation through buggy handler). +- Handler identity alone: ACL checks against handler scopes, but the handler can + reach any operation (parameterized dispatch unbounded). +- Scoped env alone: handler can only reach declared operations, but ACL is + skipped (if a declared operation requires a scope the handler doesn't have, it + still runs). + +All three together: the handler can only reach declared operations (scoped env), +those operations are ACL-checked against the handler's scoped identity (handler +identity), and internal operations are never exposed to the wire (visibility). +Principle of least privilege. + +## Consequences + +**Positive:** +- No privilege escalation through composition. A handler can only compose + operations its own identity is authorized for, and only from its declared + scope. +- Parameterized dispatch is safe. The agent/LLM tool selection case is bounded + by the scoped env — the LLM picks from the declared tool set, not from the + entire registry. The ACL checks against the handler's identity, not the + caller's. +- Buggy handlers can't accidentally escalate. A handler that tries to call an + operation outside its scoped env gets `NOT_FOUND`; one that calls an operation + its identity lacks scopes for gets `FORBIDDEN`. +- Attribution is complete. Every call carries both the caller's identity (who + initiated the chain) and the handler's identity (who is acting). The + `parent_request_id` chain traces the full agency chain. This supports the + gitea-per-agent pattern where each agent (human or LLM) has its own account. +- Session-scoped operations (alknet OQ-19) are safe by construction. They're + always `Internal`, run under the handler's identity, through the scoped env, + in a locked-down sandbox. The self-improving workflow (agents writing tools) is + bounded. +- Role-based escalation is explicit. An agent requesting promotion (session → + core) is a lower-privileged role asking a higher-privileged role (architect + with `promote` scope) to perform an action. The escalation goes through the + chain of command, not through direct authority. + +**Negative:** +- `OperationContext` has two identity fields (`identity` and + `handler_identity`), which is more complex than a single identity. This is + necessary — the principal/agent distinction is real and both are needed for + attribution and ACL. +- The assembly layer has more responsibility: it must declare each handler's + identity (scopes), its scoped composition env (which operations it may + compose), and operation visibility. This is expected — the assembly layer + assembles everything (alknet ADR-008), and forcing explicit declaration of + privilege is a feature, not a bug. +- Adding a new composition to a handler requires updating the assembly layer + (declare the new operation in the scoped env), not just the handler code. + This prevents accidental composition of unauthorized operations. +- The scoped env API is not fully specified here. The one-way constraint + (scoped env exists, is declared at registration, can be further scoped at + runtime) is fixed; the concrete API is a two-way door for implementation. + +## Assumptions + +1. **Internal calls should run under a different authority than external calls, + not skip ACL entirely.** If internal calls should skip ACL (the old `trusted` + model), this entire ADR is wrong. The assumption is that the escalation + vectors (buggy handler, parameterized dispatch) are real and must be + prevented. + +2. **Handler identity is set at registration by the assembly layer.** The + assembly layer is the trust boundary (alknet ADR-008, + [ADR-014](014-secret-material-flow-and-capability-injection.md)). If the + assembly layer is compromised, all handler identities are compromised. This + is the same trust boundary as capabilities. + +3. **The scoped env is declared at registration (static) and can be further + scoped at runtime (dynamic, for sandbox creation).** The static scoping is + the reachability control; the dynamic scoping is the sandbox boundary. If a + use case requires fully dynamic scoping (handler discovers at call time what + it can compose), the model needs extension — but the assumption is that + composition reachability is knowable at registration time. + +4. **`services/list` hides internal operations.** If internal operations should + be discoverable by remote callers (e.g., for debugging), the visibility model + needs a third state. The assumption is that internal operations are + implementation details, not part of the external API surface. + +5. **Internal operations return `NOT_FOUND`, not `FORBIDDEN`.** This prevents + existence leakage. If a use case requires distinguishing "you can't call + this" from "this doesn't exist" (e.g., for debugging), the error model needs + refinement. The assumption is that not leaking internal operation existence + is more important than debuggability from the wire. + +6. **The handler identity is a full `Identity` (with scopes), not a special + principal type.** ~~This reuses the existing `Identity` type and + `IdentityProvider` infrastructure (ADR-004).~~ **Superseded by ADR-022 + Decision 2** (alkcall ADR-018): composition authority is a declared authority + bundle (`CompositionAuthority`), not a peer `Identity`. It is not resolvable + through `IdentityProvider` and does not represent an inbound caller. The + distinction is necessary because a handler is not a network peer — its + authority is declared by the assembly layer at registration, not resolved + from credentials. + +## References + +- ADR-004: Auth as shared core (`IdentityProvider`, `Identity`) — ported to + this crate under the same number + ([004-auth-as-shared-core.md](004-auth-as-shared-core.md)) +- alknet ADR-008: Vault integration (assembly layer is the trust boundary; + alknet mono-repo ADR — the vault is an alkvault concern) +- [ADR-014](014-secret-material-flow-and-capability-injection.md): Secret + material flow and capability injection (capabilities are orthogonal — both + are set at registration by the assembly layer) +- alkcall ADR-022: Call protocol client and adapter contract (alknet ADR-017, + ported here as [ADR-017](017-call-protocol-client-and-adapter-contract.md) — + adapters produce scoped envs) +- alknet OQ-17: Abort cascade (the call tree is the agency chain — + `parent_request_id` traces principal → agent; resolved as alkcall ADR-020) +- alknet OQ-19: Session-scoped registries (session operations are always + `Internal`) +- The alkcall crate's `operation-registry.md` — the registry semantics this + model is defined against (alknet mono-repo: `crates/call/operation-registry.md`) +- The alkcall crate's `call-protocol.md` — the wire protocol whose dispatch + semantics carry the `internal` flag (alknet mono-repo: + `crates/call/call-protocol.md`) +- TypeScript `@alkdev/operations` `buildEnv()` with `allowedNamespaces` — prior + art for scoped composition env +- POC at `/workspace/toolEnv` — demonstrated the sandbox-to-registry bridge with + the full-registry exposure gap + +## Port notes + +- Renames: "alknet-http" → alkhttp; "alknet-core"/"alknet-call" → alkcall. +- Producer/consumer terminology: "wire client" → "wire consumer"; the `Visibility::External` comment now reads "call.requested from a consumer" (the call protocol's producer/consumer model — both sides of a connection can initiate). "MCP server trusted"-style inherent-directionality phrasing is not affected here (no MCP server/client mention in this ADR). The NAPI/Python/agent consumers are consumers of the call protocol, not "clients" in the connection-establishment sense. +- "future services" in the Context consumer list extended with "HTTP adapter" — the alkhttp gateway and adapters are consumers of this privilege model (adapter-registered ops are `Internal` by default per [ADR-017](017-call-protocol-client-and-adapter-contract.md)). +- Added a closing Context paragraph (new paragraph, marked as port context): why the ADR is ported to alkhttp (the gateway and adapters sit on top of these protocol semantics) and where the decision record now lives (alkcall ADR-017). This is port framing, not a new decision. +- Cross-references remapped per the verified mapping: alknet ADR-022 → alkcall ADR-018 (CompositionAuthority; ported here as [ADR-022](022-handler-registration-provenance-and-composition-authority.md)); alknet ADR-024 (env/scoped_env split) → alkcall ADR-019. ADR-004 and ADR-014 are ported to this crate under the same numbers and linked; alknet ADR-008 is cited textually (alknet mono-repo ADR; vault is an alkvault concern). +- Spec-document links (`../crates/call/operation-registry.md`, `../crates/call/call-protocol.md`) converted to textual "the alkcall crate's docs/architecture/.md" references, with the original alknet mono-repo paths noted. +- Alknet OQ numbers (OQ-15, OQ-17, OQ-19) are alknet-record citations, annotated as such — alkcall's OQ numbering differs. +- No decision content changed — the authority-switch-not-ACL-skip model, the External/Internal visibility, the scoped composition env, and the three-controls table are verbatim from the alknet ADR. \ No newline at end of file diff --git a/docs/architecture/decisions/017-call-protocol-client-and-adapter-contract.md b/docs/architecture/decisions/017-call-protocol-client-and-adapter-contract.md new file mode 100644 index 0000000..54de46d --- /dev/null +++ b/docs/architecture/decisions/017-call-protocol-client-and-adapter-contract.md @@ -0,0 +1,587 @@ +# ADR-017: Call Protocol Client and Adapter Contract + +*Ported from alknet ADR-017 (Call Protocol Client and Adapter Contract); re-targeted to alkhttp.* + +## Status + +Accepted (amended 2026-06-26, 2026-07-13, and 2026-07-16 — see "Amendments" below; the 2026-07-16 amendment per alknet ADR-089 §5 removes `CallClient::connect`) + +## Context + +The call protocol spec (alknet ADR-012, now alkcall ADR-015) defined the +stream model as bidirectional — "both sides can initiate calls." But the spec +only described the accept side: `CallAdapter` implements `ProtocolHandler`, +accepts incoming connections, and dispatches to the operation registry. The +connect side — who opens the connection, how calls are sent, how remote +operations are discovered and imported — was left as alknet OQ-15. + +The need for the connect side is concrete and immediate: + +- **Head/worker dispatch**: a head node manages worker nodes (Vast.ai, RunPod, + local Docker). The head needs to call operations on workers (exec, sync, + status) and workers need to call back (report status, request work). The + POC at `/workspace/@alkdev/dispatch` demonstrated this over SSH+axum; under + the call protocol, it's cross-node composition. +- **NAPI/Python adapters**: Node.js and Python consumers need to call operations + on an alk node. They speak the EventEnvelope wire format over a connection. +- **Agent tool dispatch**: an agent handler needs to call operations on remote + nodes (tools, services) the same way it calls local operations — through + `OperationEnv::invoke()`. The `from_call` adapter makes remote operations + appear in the local registry. +- **Cross-protocol interop**: external systems (HTTP APIs, MCP servers) are + imported via `from_openapi` and `from_mcp`. The reverse direction — + exposing local operations to external systems — needs `to_openapi` and + `to_mcp`. + +The `@alkdev/operations` TypeScript package demonstrated the adapter patterns +(`from_openapi`, `from_mcp`) and the `buildEnv` composition mechanism. The Rust +implementation defines the canonical traits (alknet ADR-013, now alkcall +ADR-033). + +alknet OQ-15 was constrained by +[ADR-014](014-secret-material-flow-and-capability-injection.md) (adapters take +credential sources, not static tokens) and +[ADR-015](015-privilege-model-and-authority-context.md) (adapter-registered +operations are `Internal` by default). This ADR locks the remaining one-way +door: the client/adapter contract architecture. The decision record now lives +in the alkcall crate (alkcall ADR-022); this port exists because alkhttp +implements the adapter contract — the `OperationAdapter` trait consumes this +crate's HTTP-backed adapter implementations, and the gateway `/search` and +`/schema` endpoints are the discovery surface `from_call` mirrors. + +## Decision + +### 1. `CallClient` opens connections and shares the dispatch loop + +`CallClient` opens a connection to a remote node with ALPN `alk/call`. Once +connected, the connection is symmetric — both sides can send and receive +`call.requested`. The `CallClient` is not just a caller; it is also a callee. +It has its own operation registry to dispatch incoming calls from the remote +side. + +```rust +pub struct CallClient { + registry: Arc, + identity_provider: Arc, +} + +impl CallClient { + pub async fn connect(&self, addr: SocketAddr, credentials: CallCredentials) -> Result; +} +``` + +The dispatch loop is shared between `CallAdapter` and `CallClient`. Once a +connection is established (whether accepted by the adapter or opened by the +client), the same logic applies: read `EventEnvelope` frames, dispatch to the +operation registry, write responses, and send outgoing `call.requested` events +for calls initiated on this side. The only difference is who opened the +connection. + +`CallConnection` provides: +- `call(operation_id, input) -> ResponseEnvelope` — send `call.requested`, + await `call.responded` (one result) +- `subscribe(operation_id, input) -> Stream` — send + `call.requested`, yield each `call.responded` until `call.completed` or + `call.aborted` +- `abort(request_id)` — send `call.aborted`, cascade to descendants (alkcall ADR-020) +- `services_list() -> Vec` — call `services/list` +- `services_schema(name) -> OperationSpec` — call `services/schema` + +### 2. Connection direction is independent of call direction + +Who opens the connection (who has the public IP, who uses a relay, who +connects out reverse-runner style) is a connection-layer concern, not a +protocol-layer concern. Once connected, both sides can call each other. + +| Topology | Who advertises | Who opens connection | Who can call whom | +|----------|---------------|----------------------|-------------------| +| Public service | Producer (public IP/domain) | Consumer | Both directions | +| P2P (iroh relay) | Both (relay-assisted) | Either | Both directions | +| Reverse (runner pattern) | Head (public IP) | Worker connects out | Both directions | +| Reverse (dispatch pattern) | Worker (public SSH port) | Head connects out | Both directions | + +The protocol does not distinguish producer and consumer after connection +establishment. The `CallAdapter` accepts connections; the `CallClient` opens +connections. Both dispatch incoming and outgoing calls through the same +mechanism. + +### 3. `from_call` adapter imports remote operations + +`from_call` does for call protocol endpoints what `from_openapi` does for HTTP +APIs: discovers operations and registers them in the local registry with +forwarding handlers. + +```rust +pub async fn from_call( + connection: &CallConnection, + config: FromCallConfig, +) -> Vec +``` + +The adapter: +1. Calls `services/list` on the remote node → gets the list of `External` + operations +2. Calls `services/schema` for each → gets the input/output JSON Schemas and + declared error_schemas ([ADR-023](023-operation-error-schemas.md)) +3. For each discovered operation, constructs a `HandlerRegistration` bundle: + - The spec mirrors the remote operation's name, namespace, type, schemas + (input, output, and error_schemas — ADR-023), and access control + - The handler sends `call.requested` through the `CallConnection` and awaits + `call.responded` (or streams for `Sub` operations) + - `provenance: FromCall`, `composition_authority: None`, `scoped_env: None` + (leaves — [ADR-022](022-handler-registration-provenance-and-composition-authority.md)) +4. The caller registers these bundles in their local registry (into the + connection's overlay — alkcall ADR-024/ADR-019) + +`from_call`-registered operations are `Internal` by default +([ADR-015](015-privilege-model-and-authority-context.md)) — they +are composition material, not directly callable from the wire. The handler +that composes them is `External`. + +The `FromCallConfig` includes: +- The credential source for the outbound connection + ([ADR-014](014-secret-material-flow-and-capability-injection.md)) — TLS identity, + auth token, or capability-provided credentials +- An optional namespace prefix (to avoid collisions when importing from + multiple remote nodes) +- An optional operation filter (to import only specific operations) + +### 4. `to_openapi` and `to_mcp` adapters export local operations + +The reverse direction — exposing local operations to external systems: + +- **`to_openapi`**: generates an OpenAPI spec from the local registry's + `External` operations. External systems (HTTP clients, API gateways) can + discover and call alk operations through a standard HTTP interface. This is + the gateway pattern — the five fixed gateway endpoints + (`/search`/`/schema`/`/call`/`/batch`/`/subscribe`), not one path per + operation (ADR-042) — implemented in alkhttp. +- **`to_mcp`**: exposes local operations as MCP tools. MCP clients (editors, + AI tools) can discover and call alk operations through the MCP protocol — + the four fixed gateway tools (ADR-041), implemented in alkhttp. + +These adapters are outbound bridges — they translate the call protocol's +operation model into external protocol formats. They do not modify the local +registry; they project it. + +### 5. The adapter contract trait + +The adapter patterns share a common shape: they produce +`HandlerRegistration` bundles that register in the local registry. The +trait: + +```rust +#[async_trait] +pub trait OperationAdapter: Send + Sync { + async fn import(&self) -> Vec; +} +``` + +The return type is `Vec` (not `(OperationSpec, +Handler)` pairs) — [ADR-022](022-handler-registration-provenance-and-composition-authority.md) +changed the registration API to the bundle +shape, and adapters must produce bundles. Adapter convenience methods +construct bundles with `composition_authority: None` and `scoped_env: None` +for the leaf ops they produce. + +The trait is **async** because `from_call` requires async discovery +(`services/list` + `services/schema` over a connection). A synchronous +trait cannot accommodate `from_call` without a separate async pre-step that +populates a cache. The sync adapters (`from_openapi`, `from_mcp` reading a +static spec) trivially satisfy an async trait — their `import()` bodies +contain no `.await` points. The async/sync question is decided: the trait +is async. + +Implementations: +- `FromOpenAPI` — imports from an OpenAPI spec (HTTP-backed handlers; + implemented in alkhttp) +- `FromMCP` — imports from an MCP server (MCP-backed handlers; implemented in + alkhttp) +- `FromCall` — imports from a remote call protocol endpoint + (call-protocol-backed handlers; implemented in alkcall) +- ~~`FromJsonSchema` — imports from a JSON Schema definition (schema-only, no + handler — used for validation or client generation)~~ — **superseded by + [ADR-066](066-from-jsonschema-as-http-adapter.md)**: `from_jsonschema` is + now an HTTP-backed single-endpoint adapter in alkhttp (reqwest + forwarding handler, not a schema-only placeholder); `FromJsonSchema` + provenance stays in alkcall as a handler-bearing leaf. + +The `to_*` adapters are outbound projections, not `OperationAdapter` +implementations — they consume the registry, they don't produce entries for it. + +The specific trait signatures (error types, configuration parameters) are +two-way doors for implementation. The one-way doors are the architectural +commitments: adapters produce `HandlerRegistration` bundles +([ADR-022](022-handler-registration-provenance-and-composition-authority.md)), the +trait is async (required by `from_call`), and the adapter *trait* lives in +alkcall while adapter *implementations* live with their transport +(HTTP-backed adapters in alkhttp per ADR-066; connection-backed `from_call` +in alkcall). See alkcall's `client-and-adapters.md` §"Adapter Location Map." + +### 6. Cross-node call tree and abort cascade + +When a `from_call` handler sends `call.requested` to a remote node, the call +participates in the local call tree via `parent_request_id`. If the parent is +aborted, the cascade (alkcall ADR-020) reaches the `from_call` handler, which sends +`call.aborted` to the remote node. The remote node cascades to its own +descendants. The abort crosses the node boundary transparently. + +``` +Head node Worker node + r1: /dispatch/run_training + r1-a: worker/exec (from_call handler) + → call.requested { id: r1-a } ────────→ receives, dispatches to exec + r1-a-1: exec spawns child + user aborts r1 + cascade to r1-a + from_call handler sends: + call.aborted { id: r1-a } ───────────→ receives, cascades to r1-a-1 + aborts exec and children +``` + +### 7. Credential sources for connections + +The `CallClient` needs credentials to authenticate to the remote node. These +come from capabilities +([ADR-014](014-secret-material-flow-and-capability-injection.md)), not environment +variables. The credential types: + +- **TLS identity**: the local node's Ed25519 key (RFC 7250 raw key) or X.509 + cert, derived from the vault at startup +- **Auth token**: an opaque token for call-protocol-level authentication, + decrypted from the vault or derived from a shared secret +- **Remote identity verification**: the expected fingerprint or cert of the + remote node, stored as a capability (not an env var or config file) + +The `from_call` adapter receives these credentials at registration time, +same as `from_openapi` receives HTTP credentials. + +## Consequences + +**Positive:** +- Cross-node composition works the same as local composition. A handler calls + `env.invoke("worker", "exec", ...)` and doesn't know (or care) whether + `worker/exec` is a local operation or a `from_call`-imported remote + operation. The composition is transparent. +- The head/worker pattern (dispatch, runners) is a connection topology, not a + protocol feature. Workers can connect to heads (runner pattern) or heads can + connect to workers (dispatch pattern) — the protocol handles both. +- `from_call` is the same pattern as `from_openapi` and `from_mcp`: discover, + register, forward. The adapter contract is unified. +- `to_openapi` and `to_mcp` enable interop with non-alk systems without + those systems needing to speak EventEnvelope. +- The abort cascade (alkcall ADR-020) crosses node boundaries transparently. No + consumer needs to implement cross-node abort propagation. +- The NAPI and Python adapters can use `CallClient` directly to call remote + operations — they don't need a separate client implementation. + +**Negative:** +- `CallClient` has its own operation registry (for dispatching incoming calls + from the remote side). This is a second registry instance, not the global + one — it needs to be populated with the operations this node wants to expose + to that specific remote peer. The specific mechanism (sharing the global + registry, a peer-scoped subset, or a separate registry) is a two-way door. +- `from_call`-registered operations have a latency cost: each invocation sends + a `call.requested` and awaits a `call.responded`. This is + inherent to remote calls and not specific to the adapter pattern. Caching + or batching strategies are consumer concerns. +- The `to_*` adapters need to translate the call protocol's operation model + (JSON Schema, EventEnvelope, subscribe/stream) into external formats + (OpenAPI paths, MCP tools). Some semantics don't map cleanly (e.g., + `Sub` streaming in OpenAPI, bidirectional calls in MCP). The adapters handle + these with best-effort mappings and document the gaps. +- **Published `to_*` specs are compatibility contracts.** The "best-effort" + mapping label is internal framing. Once a generated spec is published and + external clients build against it, the mapping semantics (e.g., + `Sub` streaming → SSE long-poll) become a de facto contract. Changing the + mapping later breaks every client. `to_*` mapping choices are two-way + *before* first publication but one-way *after*. Version the generated + specs (e.g., OpenAPI spec version tied to the registry's External + operation set version) and emit a spec version marker so consumers can + detect mapping changes. This is the "published artifact is a contract" + blind spot in alknet ADR-009's framework: it classifies doors by reversal cost + in the codebase, not by compatibility cost for external consumers. (The + versioning was subsequently specced — alknet ADR-045, ported here as + [ADR-045](045-to-openapi-gateway-spec-versioning.md).) +- **Sharing the global registry with a `CallClient` exposes local + capabilities to the remote peer.** Each `HandlerRegistration` carries + `Capabilities` with secret material. If the `CallClient` shares the + global registry, a remote peer calling an External operation triggers + dispatch that populates `OperationContext.capabilities` from the local + registration bundle — meaning the local node's API keys and signing keys + are used for the remote peer's call. A peer-scoped subset must filter by + capability remote-safety (is this operation's capability safe to expose + to this peer?), not just operation name. The registry-mechanism choice + (share global vs subset vs separate) is two-way mechanically but has a + security dimension post-[ADR-022](022-handler-registration-provenance-and-composition-authority.md): the + "share global" option is a + capability-exposure decision, not just a dispatch decision. +- The `CallConnection` abstraction adds a layer between the handler and the + raw transport stream. This is necessary for the `from_call` handler to be + transparent — it shouldn't know about the underlying transport streams, only + about call/request semantics. + +## Assumptions + +1. **The connection is symmetric after establishment.** Both sides can send + and receive `call.requested`. If a future use case requires one-directional + connections (e.g., a fire-and-forget notification where the receiver can't + call back), the model needs extension. The assumption is that bidirectional + is the correct default. + +2. **`services/list` and `services/schema` are the discovery mechanism for + `from_call`.** The remote node exposes its `External` operations through + these built-in operations. If a remote node doesn't support service + discovery (e.g., a minimal worker that only accepts specific calls), + `from_call` needs an alternative discovery mechanism (static config, manual + spec). The assumption is that nodes participating in cross-node composition + support service discovery. + +3. **The `from_call` handler is transparent to composition.** A handler that + calls `env.invoke("worker", "exec", ...)` doesn't know it's a remote call. + If the remote node is unreachable or the connection drops, the handler gets + a `call.error` (same as a local handler error). The assumption is that + remote call failures are handled the same as local handler failures. + +4. **`from_call`-registered operations mirror the remote spec.** The imported + `OperationSpec` has the same name, namespace, type, schemas (input, output, + and error_schemas per [ADR-023](023-operation-error-schemas.md)), and access + control as the remote operation. If the remote operation changes (new + schema, renamed), the imported spec is stale until re-import. The + assumption is that re-import happens on reconnection or is triggered + explicitly. Hot-swapping imported specs is a two-way door. + +5. **The `to_*` adapters are projections, not live bridges.** `to_openapi` + generates a spec; it doesn't proxy HTTP requests. An external HTTP client + calling the generated OpenAPI endpoints needs an HTTP host (alkhttp) that + translates HTTP requests into call protocol operations — the gateway + dispatch. The assumption is that `to_*` generates specs/tools, and a + separate HTTP/MCP handler bridges the actual traffic. + +## References + +- alknet ADR-005: irpc as call protocol foundation (superseded — see alkcall + ADR-014, irpc was never integrated; framing is hand-rolled) +- alkcall ADR-015: Call Protocol Stream Model (alknet ADR-012; bidirectional + streams) +- alkcall ADR-033: Rust as canonical implementation language (alknet ADR-013; + adapter traits in Rust) +- [ADR-014](014-secret-material-flow-and-capability-injection.md): Secret + material flow (credential sources, not static tokens) +- [ADR-015](015-privilege-model-and-authority-context.md): Privilege model + (adapter ops are Internal by default) +- alkcall ADR-020: Abort cascade for nested calls (alknet ADR-016; cross-node + abort propagation) +- alkcall ADR-023: Peer-Scoped Registry Filtering for CallClient Inbound + Dispatch (alknet ADR-028; resolves the §1 Consequences security dimension + flagged as a two-way door) +- alkcall ADR-024: Peer-Graph Routing Model (alknet ADR-029; supersedes + alkcall ADR-023's `remote_safe`/`trusted_peer` gate) +- alknet OQ-15: Call protocol client and adapter contract (resolved by this + ADR — the decision record is alkcall ADR-022) +- alknet OQ-25..28: Two-way-door remainders from the call-completion gap + analysis (DC-1 shape, DC-4 error type, DC-2 re-import trigger, DC-3 + namespace collision — see alknet mono-repo `open-questions.md`; the OQ + numbers are alknet-record citations) +- The alkcall crate's `call-protocol.md` — the wire protocol spec (alknet + mono-repo: `crates/call/call-protocol.md`) +- The alkcall crate's `operation-registry.md` — the registry spec (alknet + mono-repo: `crates/call/operation-registry.md`) +- The alkcall crate's `client-and-adapters.md` — the spec that operationally + fills the gap this ADR left to implementation (alknet mono-repo: + `crates/call/client-and-adapters.md`) +- alknet mono-repo `docs/research/alknet-call-completion/gap-analysis.md` — + DC-1..4, the decisions that needed resolution before implementation +- TypeScript `@alkdev/operations` — `from_openapi`, `from_mcp`, `buildEnv` + prior art +- POC at `/workspace/@alkdev/dispatch` — head/worker dispatch over SSH+axum + +## Amendments (2026-06-26) + +This ADR left four decisions as two-way doors (§1 Consequences flagged DC-1's +security dimension; §5 noted trait signatures are two-way doors; Assumption 4 +noted re-import hot-swap is a two-way door; §3 mentioned the namespace prefix). +The call-completion gap analysis (alknet mono-repo +`docs/research/alknet-call-completion/gap-analysis.md` +DC-1..4) resolved them. The resolutions: + +### DC-1 — CallClient registry scope: resolved by alkcall ADR-023, superseded by alkcall ADR-024 + +The §1 Consequences security dimension was originally resolved by alknet +ADR-028 (default-deny `remote_safe: bool` + `trusted_peer` opt-in; now +alkcall ADR-023). **alknet ADR-028 is now superseded by alknet ADR-029** +(2026-06-27; now alkcall ADR-024): the flat-namespace single-peer model +alknet ADR-028 built on cannot express the head→N-workers pattern, and the +`remote_safe`/`trusted_peer` gate duplicates the existing +`AccessControl`/`Identity` machinery while reintroducing the blanket-bypass +anti-pattern [ADR-015](015-privilege-model-and-authority-context.md) killed. +alkcall ADR-024 replaces the flat overlay with peer-keyed overlays + +`PeerRef` routing, and retires `remote_safe`/ +`trusted_peer` in favor of `AccessControl::check(peer_identity)` — the +existing authorization path that was already in the dispatch path. The peer- +scoping question this section flagged is now answered structurally (peer-keyed +overlays), not by a parallel boolean gate. + +### DC-4 — OperationAdapter trait error type: resolved + +§5 showed `async fn import(&self) -> Vec` with no error +type. The trait returns `Result, AdapterError>` +where `AdapterError` is a crate-level enum. The *presence* of the error type +is recorded in the alkcall crate's `client-and-adapters.md`; +the exact variants are the two-way-door remainder, tracked as alknet OQ-26. + +### DC-2 — from_call re-import on reconnection: manual free function + +Assumption 4 noted re-import "happens on reconnection or is triggered +explicitly." The decision is **manual**: `from_call` is a free function; the +assembly layer calls it after establishing the connection. The overlay is +per-connection (Layer 2, alkcall ADR-024/ADR-019), so re-import on reconnect is +naturally scoped; a stale overlay dies with the connection. A +`CallConnection::refresh()` method for mid-connection re-discovery is a +genuine feature addition — non-breaking, additive — if a deployment needs +manual re-discovery without drop-and-reconnect. Two-way door; recorded in +the alkcall crate's `client-and-adapters.md`; tracked as alknet OQ-27. See +alknet ADR-069 (from_call is a manual free function; alkcall ADR-028) for the +full rationale. + +### DC-3 — from_call namespace collision: default set + +§3's `FromCallConfig` namespace prefix is **optional, default no prefix, +collision = error**. A node importing from two remotes that both expose the +same unprefixed op name should fail loudly. The operator adds prefixes when +importing from multiple sources. Two-way door; recorded in the alkcall +crate's `client-and-adapters.md`; tracked as alknet OQ-28. + +### Operational spec + +The gap this ADR left to implementation — the `CallClient` API, the +`from_call` flow, the trait signature, the adapter location map, the +no-env-vars invariant, and the exchange-of-operations pattern — is +specified in the alkcall crate's `client-and-adapters.md` (alknet mono-repo: +`crates/call/client-and-adapters.md`). That document +is the operational complement to this ADR; this ADR remains the architectural +authority. + +## Amendments (2026-07-09) + +### `from_jsonschema` clause superseded by ADR-066 + +The §5 `FromJsonSchema` implementation listing ("schema-only, no handler") +is **superseded by [ADR-066](066-from-jsonschema-as-http-adapter.md)**. +`from_jsonschema` is now an HTTP-backed single-endpoint adapter in +alkhttp (reqwest forwarding handler, same shape as `from_openapi`), +not a schema-only placeholder in alkcall. The `FromJsonSchema` +provenance variant stays in alkcall (`OperationProvenance`) but is +now a handler-bearing leaf, not a "no handler" entry. The "schema-only, +no handler" concept is removed — schema validation without a handler is +served by consuming `OperationSpec` directly. The §5 "adapters live in +alkcall" one-way-door statement is corrected above to "the adapter +trait lives in alkcall; implementations live with their transport." +See [ADR-066](066-from-jsonschema-as-http-adapter.md) and the alkcall +crate's `client-and-adapters.md` §"from_jsonschema". + +## Amendments (2026-07-13) + +### `CallClient` transport-agnostic API (mirrors alknet ADR-080's amendment) + +The §1 Decision framed `CallClient::connect(addr: SocketAddr, +credentials)` as the primary constructor and described it as "opens a +QUIC connection." The operational spec +(the alkcall crate's `client-and-adapters.md`) +framed `spawn_dispatch(connection)` as the "lower-level API" that +`connect()` uses after the dial. That framing welded the +client-side one-way-door API to QUIC — the same welding alknet ADR-065 +unwound on the accept side (alkcall ADR-007) and alknet ADR-080 corrected for +`ChannelClient` (alkcall ADR-043). + +The call protocol is transport-agnostic (alkcall ADR-015 EventEnvelope +framing; alknet ADR-065 `Connection::from_stream`/`from_bidi` — alkcall +ADR-007 — accept any `AsyncRead + AsyncWrite`). The connect side is half of +that protocol and must not be coupled to a transport. This amendment reframes +the existing code (which already has the right structure — +`spawn_dispatch` is not feature-gated, `connect` is +`#[cfg(feature = "quinn")]`): + +- **`CallClient::spawn_dispatch(connection: Connection)`** — the + transport-agnostic primary constructor and the one-way-door API. + Takes a pre-established `Connection` (any transport), spawns the + shared dispatch loop, returns a live `CallConnection`. Mirrors the + accept-side `CallAdapter::handle(Connection)` and + `ChannelClient::from_connection` (alkcall ADR-043). +- **`CallClient::connect(addr, credentials)`** — ~~a QUIC convenience + constructor~~ **REMOVED per alknet ADR-089 §5 (2026-07-16; alkcall + ADR-045)**. The dial is centralized in `AlknetClient` (the native client + dial seam, alkcall ADR-045); `connect` is + deleted, not retained as a two-way-door convenience, to avoid + alkcall depending on the client-dial crate and to let alkcall + shed its TLS/transport deps. Callers compose + `AlknetClient::dial_quic(...).await?` + + `CallClient::new(...).spawn_dispatch(conn)`. + +The door-type classification is updated: `spawn_dispatch` is one-way +(the handler-facing surface); ~~`connect` is two-way (additive +convenience)~~ `connect` is **removed** (alknet ADR-089 §5). The +`AlknetClient` extraction (alknet OQ-55) is **resolved** by alknet ADR-089 — +the shared dial seam is the client-dial crate; `spawn_dispatch` is the +protocol-crate take-over that consumes the dial's `Connection`. + +See the alkcall crate's `client-and-adapters.md` +§"CallClient" for the reframed operational spec. + +## Port notes + +- Renames: "alknet-http" → alkhttp; "alknet-core"/"alknet-call" → alkcall; + "alknet node"/"non-alknet systems" → "alk node"/"non-alk systems". +- ALPN correction: `alknet/call` → `alk/call` (alkcall ADR-004 renamed the + ALPN convention to the `alk/` namespace). +- Producer/consumer terminology: "server side"/"client side" (§Context, + §1, §6) → "accept side"/"connect side" for the connection-establishment + half; "who advertises"/"who opens connection" table retained (it was + already direction-of-establishment language); §2's closing sentence now + says the protocol does not distinguish producer and consumer after + connection establishment. "Node.js and Python clients" → "consumers". + HTTP/MCP inherent directionality is untouched ("HTTP clients", "MCP + servers", "MCP clients" keep their names — those are protocol roles of the + external systems, not call-protocol roles). +- `Subscription` → `Sub` (alkcall renamed `OperationType::Subscription` to + `Sub`; alkcall ADR-046 added `OperationType::Pub` — producer→consumer + streaming via `call.published`, `HandlerKind::Sink`). §1's + `subscribe()` description, §3's "or streams for subscriptions", and the + Consequences "subscriptions → SSE long-poll" example now say `Sub`. +- §4 corrected to name the concrete alkhttp realizations: `to_openapi` is the + OpenAPI gateway pattern (ADR-042 — five fixed gateway endpoints, not one + path per operation) and `to_mcp` is the MCP tool-gateway pattern (ADR-041 — + four fixed gateway tools). The original predates those ADRs; the decision + content (outbound projections of `External` ops) is unchanged. +- Assumption 5 correction: "a separate HTTP handler (alknet-http)" → "an HTTP + host (alkhttp) ... the gateway dispatch" — the direct-call per-operation + HTTP surface was removed by ADR-047; the gateway is the sole invoke path. +- Amendment remappings (verified alknet→alkcall ADR mapping): alknet + ADR-028 → alkcall ADR-023; alknet ADR-029 → alkcall ADR-024; alknet ADR-016 + (abort cascade) → alkcall ADR-020; alknet ADR-024 (registry layering) → + alkcall ADR-019; alknet ADR-065 (`Connection::from_stream`) → alkcall + ADR-007; alknet ADR-080 (`ChannelClient`) → alkcall ADR-043; alknet ADR-089 + (dial seam) → alkcall ADR-045; alknet ADR-012 (stream model) → alkcall + ADR-015; alknet ADR-069 (from_call manual free function) → alkcall ADR-028; + alknet ADR-013 (Rust canonical) → alkcall ADR-033; alknet ADR-009 (one-way + door framework) → alkcall ADR-032. The 2026-07-13 amendment title's + "mirrors ADR-080's amendment" cites the alknet number (the amendment + mirrored the alknet-record ADR-080). +- All `client-and-adapters.md` relative links converted to textual + "the alkcall crate's `client-and-adapters.md`" references (the document + lives in the alkcall crate's docs/architecture/). +- alknet OQ numbers (OQ-15, OQ-25..28, OQ-55) are alknet-record citations, + annotated as such — alkcall's OQ numbering differs (e.g., alkcall OQ-25 is + BiStream type definition, not the DC remainders). +- The `QuicCallCredentials`-shaped `connect(addr, credentials)` signature in + §1 is retained as decision history (the 2026-07-16 amendment removes + `connect`); the QUIC-specific wording "opens a QUIC connection" is + softened to "opens a connection" in the §1 prose per the transport-agnostic + amendment, with the amendment block recording the original framing. +- Status line: "per ADR-089 §5" annotated as alknet ADR-089 (alkcall + ADR-045) — the amendment citation refers to the alknet-record number. +- No decision content changed — the shared dispatch loop, the + connection-direction independence, the adapter contract trait, the + cross-node abort cascade, the credential-source rule, and all three + amendment blocks are verbatim from the alknet ADR modulo the mechanical + corrections logged above. \ No newline at end of file diff --git a/docs/architecture/decisions/022-handler-registration-provenance-and-composition-authority.md b/docs/architecture/decisions/022-handler-registration-provenance-and-composition-authority.md new file mode 100644 index 0000000..1e1951b --- /dev/null +++ b/docs/architecture/decisions/022-handler-registration-provenance-and-composition-authority.md @@ -0,0 +1,705 @@ +# ADR-022: Handler Registration, Provenance, and Composition Authority + +*Ported from alknet ADR-022 (Handler Registration, Provenance, and Composition Authority); re-targeted to alkhttp.* + +## Status + +Accepted + +## Context + +[ADR-015](015-privilege-model-and-authority-context.md) established the +privilege model: the `internal` flag marks +composition-originated calls and switches the ACL from the caller's identity +to the handler's identity. This replaces the old `trusted: bool` flag, which +skipped ACL entirely — a privilege escalation vector. The core decision in +ADR-015 is sound: internal calls switch authority, they don't skip ACL. + +However, ADR-015 left three things unspecified, which the pre-implementation +review (docs/reviews/001-pre-implementation-architecture-sanity-check.md, +findings C1–C4) identified as critical gaps: + +1. **`handler_identity` has no registration path.** ADR-015 says the handler's + identity is "set at registration by the assembly layer" (Assumption 2) and + that "ACL check runs against the handler's identity (set at registration)" + (Decision 1). But the registration API shown in operation-registry.md — + `register(spec, handler)` and `OperationRegistryBuilder::with(spec, + handler)` — accepts no identity. Tracing the dispatch path reveals that + `build_root_context` sets `handler_identity: None` for wire calls (correct + for the root), and `OperationEnv::invoke()` propagates + `parent.handler_identity.clone()` to children. Since the root's + `handler_identity` is `None`, every internal call gets `handler_identity: + None` — meaning ADR-015's "ACL runs against `handler_identity` for internal + calls" checks against `None`, which is the privilege-escalation gap ADR-015 + was written to close. + +2. **The scoped composition env has no registration/construction path.** + ADR-015 says the `OperationEnv` given to a handler is "scoped — it can + only invoke a declared set of operations, set at registration by the + assembly layer" (Decision 4, Assumption 3). But `register(spec, handler)` + takes no scoped-env declaration, `OperationSpec` has no field for it, and + the only `OperationEnv` implementation shown is `LocalOperationEnv` wrapping + the *full* registry — no scoping layer exists. + +3. **`Capabilities` lives in two unconnected models.** + [ADR-014](014-secret-material-flow-and-capability-injection.md) and + operation-registry.md show two models for how a handler gets outbound + credentials: construction-time capture in the handler closure (Model A) and + per-request on `OperationContext.capabilities` propagated through + composition (Model B). The two don't connect: if the handler closure + captured capabilities at construction, `OperationContext.capabilities` is + either redundant or must be populated from the closure — but the closure + receives the context, it isn't passed it. An implementer would have to + invent the bridge, and the consuming crates (call, agent, napi) could + diverge. + +Beyond these wiring gaps, there is a deeper issue with ADR-015's Assumption 6: +"the handler identity is a full `Identity` (with scopes), not a special +principal type." `Identity` was designed for **inbound peer identity** — who +is calling me from the network. A handler is not a peer. Its `id` field would +be something like `"agent-chat-handler"` — a label, not something resolvable +through `IdentityProvider`. Calling it an `Identity` implies it's a peer, +which it isn't. It's an authority bundle. + +### The kernel/user analogy + +This is structurally the same problem an operating system solves with +kernel/user mode: + +- User calls `getaddrinfo()` — the syscall gate (an **External** op). The + kernel checks the user's capabilities at entry. +- `getaddrinfo` internally makes DNS queries, allocates sockets, reads + `/etc/hosts` — **Internal** kernel functions. They don't check the user's + `CAP_NET_RAW`. They run under **kernel authority**. +- The user does NOT need `CAP_NET_RAW` to resolve DNS. The kernel does network + access on the user's behalf, under the kernel's own authority. + +The key principle: **the user's authority is checked once at the gate. Inside, +the handler runs under its own authority. The user's authority does not +propagate into internal calls.** + +This is exactly what ADR-015 specifies. The `internal` flag is the boundary +crossing. When `internal: true`, ACL switches from the caller's identity to +the handler's composition authority. The user's `[chat]` scope got them through +`/agent/chat`'s External ACL. Once inside, it's `/agent/chat`'s composition +authority that authorizes composing `/vastai/listMachines` — not the user's. + +### The graph framing + +Call trees and operation registries are graph-shaped. The TypeScript +`@alkdev/flowgraph` package models this explicitly with three graphs: + +1. **Operation Graph** (static) — nodes are registered operations, edges are + type-compatibility relationships. Built from `OperationSpec`s at startup. +2. **Call Graph** (dynamic) — nodes are call invocations (request IDs), edges + are parent-child relationships (`parent_request_id`). Built from call + protocol events at runtime. +3. **Scoped Operation Subgraph** (per-handler, static) — the declared subset + of the operation graph that a handler may reach. This is what ADR-015 calls + the "scoped env," framed as a subgraph rather than a list of names. + +This ADR uses the graph *model* as structural framing but does not mandate a +graph *library*. For v1, the operation graph can be implicit (a +`HashMap`), the call graph can be implicit (the +`PendingRequestMap` indexed by `parent_request_id` *is* a call graph), and the +scoped env can be a `HashSet` of reachable operation names. A +dedicated flowgraph crate (or folding graph structures into alkcall) is a +future enhancement for workflow templates, type compatibility validation, and +call-graph observability — not a prerequisite for the security model. + +## Decision + +### 1. Provenance is the primary registration axis + +Every registered operation carries a provenance tag that classifies where it +came from. Provenance determines whether the operation can compose, whether it +has composition authority, its default visibility, and its trust model. + +```rust +pub enum OperationProvenance { + /// Assembly-written, trusted code, can compose. + Local, + /// HTTP forwarding stub (from_openapi), leaf — cannot compose. + FromOpenAPI, + /// MCP forwarding stub (from_mcp), leaf — cannot compose. + FromMCP, + /// Call-protocol forwarding stub (from_call). Leaf in the local registry — + /// forwards calls to a remote node; cannot compose locally. + FromCall, + /// HTTP forwarding stub (from_jsonschema, single endpoint), leaf — + /// cannot compose. (ADR-066: was "no handler — schema only"; now a + /// real reqwest-backed forwarding handler in alkhttp.) + FromJsonSchema, + /// Agent-written, sandboxed, can compose within sandbox bounds. + Session, +} +``` + +| Provenance | Can compose? | Has composition authority? | Default visibility | Trust model | +|-----------|-------------|---------------------------|-------------------|-------------| +| `Local` | Yes | Yes — scopes set by assembly layer | External or Internal (assembly declares) | Trusted code | +| `FromOpenAPI` | No (leaf) | No | Internal | HTTP endpoint trusted; handler is a forwarding stub | +| `FromMCP` | No (leaf) | No | Internal | MCP server trusted; handler is a forwarding stub | +| `FromCall` | No (leaf in local registry) | No | Internal | Remote node trusted; handler is a forwarding stub | +| `FromJsonSchema` | No (leaf) | No | Internal | HTTP endpoint trusted; handler is a forwarding stub (ADR-066) | +| `Session` | Yes (within sandbox) | Yes — scopes set by assembly layer at sandbox creation | Internal always | Untrusted code in sandbox | + +> **ADR-066 amendment (2026-07-09).** The `FromJsonSchema` row +> previously read "N/A (no handler) / N/A / N/A" — `from_jsonschema` +> was a schema-only placeholder with a +> `NOT_FOUND`-returning handler. +> [ADR-066](066-from-jsonschema-as-http-adapter.md) moved the adapter +> to alkhttp as a real HTTP-backed single-endpoint adapter with a +> reqwest forwarding handler. `FromJsonSchema` is now a leaf, same +> trust model as `FromOpenAPI` (HTTP endpoint trusted; handler is a +> forwarding stub). The "schema-only, no handler" concept is removed. + +Only `Local` and `Session` ops get composition authority. Leaves +(`FromOpenAPI`, `FromMCP`, `FromCall`, `FromJsonSchema`) don't compose, so +they don't get one. The assembly layer does not invent identities for leaves. + +### 2. Composition authority replaces `handler_identity: Identity` + +ADR-015's Assumption 6 said "the handler identity is a full `Identity` (with +scopes), not a special principal type." This ADR refines that: composition +authority is a declared authority bundle, not a peer `Identity`. It's only set +for ops that can compose (`Local`, `Session`). Leaves don't have one. + +```rust +/// Authority under which a handler composes child operations. +/// +/// This is NOT a peer `Identity` — it's not resolvable through +/// `IdentityProvider` and doesn't represent an inbound caller. It's the +/// declared authority (scopes + resources + label) that the assembly layer +/// grants a handler for composition. When the handler composes children via +/// `OperationEnv::invoke()`, the child's ACL runs against this authority, +/// not the caller's identity and not as a blanket skip. +/// +/// Only ops that can compose (`Local`, `Session`) have one. Leaves +/// (`FromOpenAPI`, `FromMCP`, `FromCall`) have `None`. +pub struct CompositionAuthority { + /// Human-readable label for attribution and logging + /// (e.g., "agent-chat", "fs-handler"). Not a peer id — not resolvable + /// through IdentityProvider. + pub label: String, + + /// Scopes the handler operates under for composition. When the handler + /// composes a child via `env.invoke()`, the child's ACL checks against + /// these scopes. Least privilege: the assembly layer grants only the + /// scopes the handler needs for its declared composition. + pub scopes: Vec, + + /// Named resource lists, same shape as `Identity.resources`. Optional. + /// e.g., {"service": ["vastai", "github"]} bounds which services the + /// handler can reach in composition. + pub resources: HashMap>, +} + +impl CompositionAuthority { + /// `None` — for leaves that don't compose (convenience for + /// `composition_authority: CompositionAuthority::none()`). + pub fn none() -> Option { None } + + /// Construct a composition authority with the given label and scopes. + pub fn new( + label: &str, + scopes: impl IntoIterator, + ) -> Self { + Self { + label: label.to_string(), + scopes: scopes.into_iter().collect(), + resources: HashMap::new(), + } + } + + /// Convert to a synthetic `Identity` for ACL matching on child calls. + /// + /// When a handler composes a child via `env.invoke()`, the child's + /// `identity` (the caller identity for ACL) is set to the parent's + /// composition authority converted to an `Identity`. This constructs + /// a synthetic `Identity { id: label, scopes, resources }` that is + /// **not** resolvable via `IdentityProvider` — it's not a peer + /// identity, it's a declared authority bundle used directly for ACL + /// matching. This creates a second `Identity` construction path (the + /// first is `IdentityProvider::resolve_*`), which is acknowledged and + /// intentional: the composition authority is a declared authority, not + /// a resolved credential. + /// + /// Returns `None` when the authority is `None` (leaf case — leaves + /// don't compose, so `as_identity()` is never called on them in + /// practice, but the `Option` makes the types line up). + pub fn as_identity(&self) -> Option { + Some(Identity { + id: self.label.clone(), + scopes: self.scopes.clone(), + resources: self.resources.clone(), + }) + } +} +``` + +This supersedes ADR-015's Assumption 6. ADR-015's core decision (authority +switch, not ACL skip) holds unchanged — the only change is *what* the +authority is and which ops have it. + +### 3. The scoped env is a declared subgraph (reachability control) + +The scoped composition env from ADR-015 is the **reachability control**: it +bounds which operations a handler can reach via `env.invoke()`. ADR-015 +specifies it as "a declared set of operations, set at registration by the +assembly layer." This ADR makes the registration path explicit and frames it +as a subgraph of the operation graph. + +```rust +/// The set of operations a handler may reach via `env.invoke()`. +/// +/// This is the reachability control from ADR-015: a handler (or an LLM +/// picking tools, or a quickjs sandbox) can only compose declared operations, +/// not the entire registry. Set at registration by the assembly layer for +/// composing ops (`Local`, `Session`). `None` for leaves — they don't +/// compose, so they get an empty/no-op env. +/// +/// Conceptually a subgraph of the operation graph. For v1, implemented as a +/// set of operation names — the *model* is a subgraph (which nodes this +/// handler can reach), but type-compatibility edges between those nodes are +/// a future enhancement for static validation, not a v1 requirement. +/// +/// The `allowed_operations` field is **private** (not `pub`). Construction +/// is via `ScopedOperationEnv::new(ops)` or `ScopedOperationEnv::empty()`. +/// Reachability is queried via `allows(&name)`. This encapsulation makes the +/// future subgraph refactor (from `HashSet` to a typed subgraph) a +/// non-breaking change to construction sites (review #002 W21). The +/// `HashSet` representation does not support type-compatibility +/// validation — session-scoped ops (alknet OQ-19, untrusted code) compose +/// without static type checking until a flowgraph crate is built. +pub struct ScopedOperationEnv { + allowed_operations: HashSet, +} + +impl ScopedOperationEnv { + /// Empty set — for leaves that don't compose (no reachable operations). + pub fn empty() -> Self { + Self { allowed_operations: HashSet::new() } + } + + /// Construct from an iterable of operation names. + pub fn new(ops: impl IntoIterator) -> Self { + Self { allowed_operations: ops.into_iter().collect() } + } + + /// Returns true if the given operation name is reachable. + pub fn allows(&self, name: &str) -> bool { + self.allowed_operations.contains(name) + } +} +``` + +### 4. The registration bundle carries all three + +The three controls from ADR-015 (visibility, composition authority, scoped +env) plus the capability injection from +[ADR-014](014-secret-material-flow-and-capability-injection.md) all enter the +system at the +same boundary: the assembly layer hands the registry a `(spec, handler)` pair +*plus* the handler's runtime context material. This ADR makes that explicit +as a registration bundle. + +```rust +pub struct HandlerRegistration { + pub spec: OperationSpec, + pub handler: Handler, + pub provenance: OperationProvenance, + /// Composition authority for this handler. `None` for leaves + /// (`FromOpenAPI`, `FromMCP`, `FromCall`) — they don't compose. + /// `Some(...)` for `Local` and `Session` ops that can compose children. + pub composition_authority: Option, + /// Scoped composition env. `None` for leaves — they get an empty + /// no-op env. `Some(...)` for composing ops. + pub scoped_env: Option, + /// Outbound credentials the handler may use (decrypted API keys, signing + /// keys, HTTP tokens). Populated by the assembly layer from the vault + /// at handler construction. See ADR-014. + pub capabilities: Capabilities, +} +``` + +The registry's `register` and builder's `with` accept a `HandlerRegistration`, +not a bare `(OperationSpec, Handler)` pair: + +```rust +impl OperationRegistry { + pub fn register(&mut self, registration: HandlerRegistration); +} + +impl OperationRegistryBuilder { + pub fn with(mut self, registration: HandlerRegistration) -> Self; +} +``` + +Adapter convenience methods (`from_openapi`, `from_mcp`, `from_call`) +construct `HandlerRegistration` with `composition_authority: None` and +`scoped_env: None` for the leaf ops they produce — the adapter doesn't grant +composition authority, and the assembly layer doesn't have to invent values +for leaves. + +### 5. The dispatch path reads from the registration bundle + +The CallAdapter's `build_root_context` and `OperationEnv::invoke()` read +composition authority, scoped env, and capabilities from the registration +bundle, looked up by operation name. + +**`build_root_context` (wire-originated call, `internal: false`):** + +```rust +fn build_root_context( + &self, + request_id: String, + operation_name: &str, // looked up in registry + identity: Option, // resolved per-request from AuthContext/auth_token +) -> OperationContext { + let registration = self.registry.registration(operation_name); + OperationContext { + request_id, + parent_request_id: None, + identity, // caller's identity (inbound — gate credential) + handler_identity: registration.composition_authority, // C1: from bundle, None for leaves + capabilities: registration.capabilities.clone(), // C3: from bundle + metadata: HashMap::new(), + abort_policy: AbortPolicy::default(), // abort-dependents (alkcall ADR-020 Decision 6) + // env/scoped_env split by ADR-024: scoped_env is the reachability + // data (from the bundle), env is the dispatch trait object (composed + // per-call by the CallAdapter from active overlays). + scoped_env: registration.scoped_env.clone() + .unwrap_or_else(ScopedOperationEnv::empty), // C2: from bundle, empty for leaves + env: self.compose_root_env(/* connection, session */), // Arc — see ADR-024 + internal: false, // wire call — ACL against caller identity + } +} +``` + +ACL for the root checks against `identity` (the caller's identity, resolved +per-request). `handler_identity` is on the context for *propagation* to +children, not for the root's own ACL. + +**`OperationEnv::invoke()` (composition-originated call, `internal: true`):** + +```rust +async fn invoke(&self, namespace: &str, operation: &str, input: Value, + parent: &OperationContext) -> ResponseEnvelope { + let name = format!("{namespace}/{operation}"); + + // Reachability check (C2): is this op in the parent's scoped env? + // If not, return NOT_FOUND. This is the reachability control. + // (ADR-024: the reachability check consults parent.scoped_env, not + // parent.env — env is now the dispatch trait, scoped_env is the data.) + if !parent.scoped_env.allows(&name) { + return ResponseEnvelope::not_found(name); + } + + let registration = self.registry.registration(&name); + let context = OperationContext { + request_id: generate_request_id(), + parent_request_id: Some(parent.request_id.clone()), + identity: parent.handler_identity.as_identity(), // parent's authority becomes the caller + handler_identity: registration.composition_authority.clone(), // C1: child's own authority + capabilities: parent.capabilities.clone(), // C3: propagate through composition + metadata: HashMap::new(), // fresh — does NOT propagate (ADR-014) + abort_policy: parent.abort_policy.clone(), // inherit parent's policy (alkcall ADR-020 Decision 6, W19) + // env/scoped_env split by ADR-024: + scoped_env: registration.scoped_env.clone() + .unwrap_or_else(ScopedOperationEnv::empty), // C2: child's own scoped env + env: parent.env.clone(), // child inherits parent's composite env (Arc::clone) + internal: true, // composition — ACL against handler_identity + }; + self.registry.invoke(&name, input, context).await +} +``` + +Two things happen here: + +1. **Reachability check**: before constructing the child context, `invoke()` + checks whether the requested op is in the parent's scoped env. If not, + `NOT_FOUND`. This bounds the parameterized-dispatch attack surface — a + handler (or an LLM picking tools) can only reach declared ops. + +2. **Authority propagation**: the child's `identity` is the parent's + `handler_identity` (the parent's composition authority becomes the caller + for the child). The child's `handler_identity` is the *child's own* + registration's `composition_authority` — so if the child itself composes + further, its children inherit the child's authority. This is the + principal/agent chain from ADR-015, now wired. + +ACL for the child checks against `handler_identity` (the child's composition +authority). For leaves, `handler_identity` is `None` — but leaves don't +compose, so their `handler_identity` is never used for ACL on a grandchild. +Leaves only have ACL checked against *themselves* (as the target of +composition), where the check is: does the parent's composition authority +satisfy the leaf's `AccessControl`? + +### 6. Capabilities are per-request, populated from the bundle (Model A reconciled) + +This ADR resolves the C3 ambiguity by adopting option (a) from the review: +capabilities are only per-request on `OperationContext`, populated by the +dispatch path from the per-handler capabilities in the registration bundle. +The construction-time "baking" described in +[ADR-014](014-secret-material-flow-and-capability-injection.md) populates the +registration bundle's `capabilities` field — the handler closure does not +capture capabilities. + +```rust +// Assembly layer: construct registration with capabilities from vault +let google_api_key = vault.decrypt(&google_key_blob)?; +let agent_registration = HandlerRegistration { + spec: agent_chat_spec(), + handler: Arc::new(agent_chat_handler), // closure captures nothing + provenance: OperationProvenance::Local, + composition_authority: Some(CompositionAuthority { + label: "agent-chat".into(), + scopes: vec!["llm:call".into(), "fs:read".into(), "vastai:query".into()], + resources: HashMap::new(), + }), + scoped_env: Some(ScopedOperationEnv::new( + ["fs/readFile", "vastai/listMachines", "llm/generate"])), + capabilities: Capabilities::new() + .with_api_key("google", google_api_key), // C3: in the bundle, not the closure +}; +``` + +The handler reads `context.capabilities` at call time. The dispatch path +populates it from `registration.capabilities`. Composition propagates it via +`parent.capabilities.clone()` in `invoke()`. No circular dependency, no +redundant models. + +### 7. The three controls together (ADR-015's model, now wired) + +| Control | What it gates | Where it's set | Without it | +|---------|--------------|----------------|-----------| +| Visibility (External/Internal) | Whether the op is callable from the wire | `OperationSpec.visibility` | Internal ops exposed to external callers | +| Composition authority | What authority internal calls run under | `HandlerRegistration.composition_authority` | ACL skipped or caller's scopes propagated (escalation) | +| Scoped env | What ops a handler can reach | `HandlerRegistration.scoped_env` | Handler can call anything in the registry (confused deputy) | + +All three enter at registration. All three reach the dispatch path via the +registration bundle. The user's identity is the **gate credential** — checked +once at the External boundary. The composition authority is the **internal +credential** — used for all composition inside. The scoped env is the +**reachability boundary** — what the handler can even attempt to compose. + +### 8. No intersection semantics + +The user's authority does NOT limit internal calls. If the user has `chat` but +not `vastai:query`, `/agent/chat` composing `/vastai/listMachines` is NOT +denied because the user lacks `vastai:query`. The user's authority was +checked at the gate (`/agent/chat` requires `chat`, user has `chat`). Inside, +the handler runs under its own composition authority. The user's authority +does not propagate into internal calls. + +This is the kernel/user model: `getaddrinfo` doesn't require the caller to +have `CAP_NET_RAW` to make DNS queries. The curated entry point exists +*because* it does things the user can't, on the user's behalf, under its own +authority. + +If a handler *wants* to act on behalf of the user (e.g., a database proxy +that runs queries under the user's DB identity), that's a **handler-level +decision** — it reads `context.identity` and explicitly narrows its +behavior. That's delegated access, not automatic intersection. The system +shouldn't silently intersect; the handler should explicitly delegate. + +## Consequences + +**Positive:** + +- The privilege model in ADR-015 is now implementable as specified. The + composition authority, scoped env, and capabilities all have registration + paths and dispatch-path wiring. No implementer has to invent the bridge. +- Leaves (`from_openapi`, `from_mcp`, `from_call`) don't get fake identities. + The assembly layer doesn't have to invent `Identity { id: + "vastai-listmachines-handler", scopes: [], resources: {} }` for forwarding + stubs that will never compose. `composition_authority: None` is natural for + leaves, not an oversight. +- External services can't self-grant composition authority. The OpenAPI spec + defines the operation interface (name, schemas, access control). The + *provenance* is set by the assembly layer when it runs `from_openapi`. The + *composition authority* is `None` for imported ops — the external service + can't grant itself scopes to compose into your registry. The assembly layer + is the sole grantor, and only for `Local` and `Session` ops. +- Capabilities have one model: per-request on `OperationContext`, populated + from the registration bundle. No closure-capture vs context duplication + ambiguity. The three consuming crates (call, agent, napi) can't diverge + because there's one wiring path. +- The graph model provides a precise structural framing without mandating a + graph library for v1. The operation graph, scoped subgraph, and call graph + are concepts that guide the API shape; HashMaps and HashSets are the v1 + implementation. A future flowgraph crate can reify these as + petgraph structures when workflow templates and type-compatibility + validation are needed. +- The kernel/user analogy makes the security model legible. The user's + authority is the gate credential (checked once at External entry). The + composition authority is the internal credential (used for all + composition inside). The scoped env is the reachability boundary (what the + handler can attempt to compose). This is the same model every OS uses, and + it's been battle-tested. + +**Negative:** + +- The registration API changes from `register(spec, handler)` to + `register(HandlerRegistration)`. This is a breaking change to the API + surface shown in operation-registry.md, but since no implementation exists + yet, it's a spec edit, not a migration. +- `CompositionAuthority` is a new type, distinct from `Identity`. This adds a + type to the call crate. It's not a peer identity — it's a declared authority + bundle. The distinction from `Identity` is intentional and necessary (a + handler is not a network peer), but it means the codebase has two + scope-bearing types. Mitigated: they serve different roles and don't + converge — `Identity` is inbound (resolved from credentials via + `IdentityProvider`), `CompositionAuthority` is declared (set by the + assembly layer at registration). +- The assembly layer has more registration-time responsibility: it must + declare each handler's provenance, composition authority, and scoped env. + This is expected — the assembly layer assembles everything (alknet + ADR-008), and forcing explicit declaration of privilege is a feature, not a + bug. An `OperationRegistryBuilder` convenience API can reduce boilerplate + for common cases (e.g., `.with_local(spec, handler, authority, env, + capabilities)` vs `.with_leaf(spec, handler, capabilities)`). +- The dispatch path does a registry lookup per call (to fetch the + registration bundle's composition authority, scoped env, and capabilities). + This is a `HashMap` lookup — negligible cost. The alternative (baking + everything into the handler closure) creates the C3 ambiguity. The lookup + is the right trade. + +**Validation strategy:** + +The security model should be validated by fuzzing. A fuzzer that generates +call trees (valid and invalid compositions, different provenance mixes, edge +cases around the gate) and asserts "no path through the call graph lets a +user with scope X reach an operation requiring Y without going through a gate +that checks X" would catch the class of privilege-escalation bug this ADR is +designed to prevent. The typebox-rs fake data generator can produce valid and +invalid inputs from JSON Schemas; with minor edits it can output invalid +inputs or a mix of valid/invalid, enabling property-based testing of the ACL +model. This is a downstream concern — the spec needs to be right first, then +the fuzzer validates the implementation against the spec. + +## Assumptions + +1. **Internal calls should run under a different authority than external + calls, not skip ACL entirely.** Inherited from ADR-015. The escalation + vectors (buggy handler, parameterized dispatch) are real and must be + prevented. + +2. **Provenance is knowable at registration time.** The assembly layer knows + whether an op is `Local`, `FromOpenAPI`, `FromMCP`, `FromCall`, or + `Session` when it registers the op — the adapter that produced the + `(OperationSpec, Handler)` pair knows its own type. If a future use case + requires provenance to be discovered at call time, the model needs + extension. + +3. **Composition reachability is knowable at registration time.** The + assembly layer can declare which operations a handler may compose when it + registers the handler. If a use case requires fully dynamic scoping + (handler discovers at call time what it can compose), the model needs + extension — but the assumption is that composition reachability is + knowable at registration time for `Local` ops, and at sandbox creation + time for `Session` ops. + +4. **The assembly layer is the trust boundary.** The assembly layer declares + provenance, composition authority, and scoped env. If the assembly layer + is compromised, all handler authority is compromised. This is the same + trust boundary as alknet ADR-008 and + [ADR-014](014-secret-material-flow-and-capability-injection.md). + +5. **Leaves don't compose.** `FromOpenAPI`, `FromMCP`, and `FromCall` ops are + forwarding stubs — they take input, forward it (over HTTP, MCP, or the + call protocol), and return output. They don't call `env.invoke()`. If a + future use case requires an imported op to compose (e.g., a `from_call` + op that locally composes other ops before forwarding), its provenance + would need to change to `Local` (it's no longer a pure forwarding stub), + or the model needs a hybrid provenance. + +6. **`Session` ops compose under restricted authority.** Session ops + (agent-written, alknet OQ-19) get composition authority scoped down by the + parent handler at sandbox creation (ADR-015's "dynamic scoping at sandbox + creation"). The assembly layer grants the sandbox's parent handler a + composition authority; the parent handler scopes it down further when + creating the sandbox. The session op's composition authority is a subset + of the parent's. + +## References + +- [ADR-014](014-secret-material-flow-and-capability-injection.md): Secret + material flow and capability injection (capabilities are + orthogonal to identity — both set at registration; this ADR specifies the + registration path ADR-014 left as a two-way door) +- [ADR-015](015-privilege-model-and-authority-context.md): Privilege model + and authority context (this ADR refines + Assumption 6 — composition authority is not a peer `Identity`; and wires + the three controls that ADR-015 specified but left without registration + paths) +- alkcall ADR-020: Abort cascade for nested calls (alknet ADR-016; the call + graph is the abort cascade tree; `parent_request_id` indexes it) +- [ADR-017](017-call-protocol-client-and-adapter-contract.md): Call protocol + client and adapter contract (adapter-registered + ops are `Internal` by default; this ADR's provenance makes that explicit; + the decision record is alkcall ADR-022) +- alkcall ADR-019: Operation registry layering (alknet ADR-024; amends this + ADR's Decision 5: the + `env` field shown in `build_root_context` and `invoke()` is split into + `scoped_env: ScopedOperationEnv` (reachability data, populated from the + bundle's `scoped_env`) and `env: Arc` + (dispatch trait object). The split is required by the overlay model — the + trait-object design is what enables connection and session overlays + to compose. The `HandlerRegistration` bundle shape, provenance model, + composition authority, and capability injection specified by this ADR + are unchanged.) +- alknet ADR-008: Vault integration point (assembly layer is the trust + boundary; alknet mono-repo ADR — the vault is an alkvault concern) +- alknet OQ-19: Session-scoped operation registries (session ops are + `Session` provenance, always `Internal`, compose under restricted + authority) +- alknet mono-repo `docs/reviews/001-pre-implementation-architecture-sanity-check.md` + (findings C1–C4, which this ADR resolves) +- alknet mono-repo `docs/reviews/002-pre-implementation-architecture-sanity-check.md` + (finding C6, resolved by ADR-024's `env`/`scoped_env` split — alkcall + ADR-019) +- `/workspace/@alkdev/flowgraph/README.md` — operation graph, call graph, and + scoped subgraph concepts (the graph model this ADR uses as framing) +- `/workspace/@alkdev/alknet-main/docs/architecture/flowgraph.md` — prior + Rust speccing of flowgraph (incomplete; this ADR uses the model, not the + crate) +- Kernel/user mode analogy: `getaddrinfo` runs under kernel authority, not + the caller's `CAP_NET_RAW`; the curated entry point exists to do things + the user can't, on the user's behalf + +## Port notes + +- Renames: "alknet-http" → alkhttp; "alknet-core"/"alknet-call" → alkcall. +- Cross-reference remappings (verified alknet→alkcall ADR mapping): alknet + ADR-016 (abort cascade) → alkcall ADR-020; alknet ADR-024 (registry + layering) → alkcall ADR-019; alknet ADR-017 (adapter contract) → alkcall + ADR-022 (ported here as ADR-017). ADR-014 and ADR-015 are ported to this + crate under the same numbers and linked. +- `from_jsonschema` correction (per alkcall ADR-027 / this crate's ADR-066): + the `FromJsonSchema` enum-doc comment and the ADR-066 amendment block now + say the adapter lives in **alkhttp** (the original said `alknet-http`, which + is the same crate post-extraction — updated to the current name rather than + left as a stale reference). The superseded-ADR citation inside the + alkcall-crate copy of this ADR cites alkcall ADR-027; the alknet-record + original cited alknet ADR-066. +- "QUIC forwarding stub (from_call)" → "Call-protocol forwarding stub + (from_call)" — the call protocol is transport-agnostic (alkcall ADR-007); + the original wording predates the transport-agnostic connection model. + Assumption 5's "over HTTP, MCP, or QUIC" → "over HTTP, MCP, or the call + protocol". +- Review paths (`docs/reviews/...`) are alknet mono-repo artifacts; annotated + as such rather than converted to links (the reviews are not ported to + alkhttp). +- alknet OQ-19 is an alknet-record citation (alkcall's OQ numbering differs). +- alknet ADR-008 is cited textually (alknet mono-repo ADR; the vault is an + alkvault concern). +- The flowgraph references are kept as absolute workspace paths (the + directories still exist); "a dedicated `alknet-flowgraph` crate" → "a + dedicated flowgraph crate" (hypothetical crate name genericized, matching + the alkcall-crate copy of this ADR). +- No decision content changed — the provenance model, `CompositionAuthority`, + `ScopedOperationEnv`, the `HandlerRegistration` bundle, the dispatch-path + wiring, the no-intersection rule, and the ADR-066 amendment are verbatim + from the alknet ADR modulo the corrections logged above. \ No newline at end of file diff --git a/docs/architecture/decisions/023-operation-error-schemas.md b/docs/architecture/decisions/023-operation-error-schemas.md new file mode 100644 index 0000000..a9d599b --- /dev/null +++ b/docs/architecture/decisions/023-operation-error-schemas.md @@ -0,0 +1,479 @@ +# ADR-023: Operation Error Schemas + +*Ported from alknet ADR-023 (Operation Error Schemas); re-targeted to alkhttp.* + +## Status + +Accepted (amended by alknet ADR-049 — protocol-level code list extended to six; alkcall ADR-021) + +## Context + +The `OperationSpec` in the call crate (now alkcall) has `input_schema` and +`output_schema` but no `error_schemas`. The `call.error` payload (the alkcall +crate's `call-protocol.md`) carries a `code` and `message`, where `code` is +one of six infrastructure codes: `NOT_FOUND`, `FORBIDDEN`, `INVALID_INPUT`, +`INVALID_OPERATION_TYPE`, `INTERNAL`, `TIMEOUT`. + +These six codes cover **protocol-level failures** — the call protocol +itself can always fail to find an operation, deny access, reject bad input, +reject the wrong dispatch method for the operation type, time out, or hit +an internal error. They are emitted by the dispatch machinery (the registry, +the adapter), not by operation handlers. `INVALID_OPERATION_TYPE` was added +by alknet ADR-049 (streaming handler for subscriptions — now alkcall ADR-021; +`invoke()` called on a `Sub`, or `invoke_streaming()` on a `Query`/`Mutation`). + +But operations also have **domain-level failures** that are not covered: + +- `/fs/readFile` can fail because the file doesn't exist, the path is + invalid, or the caller lacks OS-level read permission. These are + operation-specific failures distinct from the protocol-level + `INVALID_INPUT` (schema mismatch) or `FORBIDDEN` (scope mismatch). +- `/vastai/createMachine` can fail because the account has insufficient + credits, the machine type is unavailable in the requested region, or the + upstream API rate-limited the request. +- `/agent/chat` can fail because the LLM provider returned an error, the + context window overflowed, or the model refused the request. + +Today, these failures collapse into `INTERNAL` with a `message` string. +A consumer calling `/fs/readFile` has no way to know from the schema that it +might return `FILE_NOT_FOUND` vs `PERMISSION_DENIED` vs `INVALID_PATH`. The +caller has to parse `message` strings — the exact anti-pattern that typed +RPC is meant to avoid. This is a **type safety gap**: inputs and outputs are +typed, but errors are untyped strings. + +### Why this matters for adapters + +OpenAPI specs naturally include error information — response status codes +with schemas (e.g., `404: { schema: NotFoundError }`, `422: { schema: +ValidationError }`). MCP tool definitions carry error descriptions. The +`from_openapi` adapter ([ADR-017](017-call-protocol-client-and-adapter-contract.md)) +imports operations and mirrors +"the remote operation's name, namespace, type, schemas, and access control" +— but with no error schema field, error responses from the OpenAPI source +are dropped on import. `to_openapi` has nowhere to project error information +to. The same gap applies to `from_mcp`/`to_mcp`. + +An OpenAPI operation that declares: + +```yaml +responses: + '200': { schema: MachineList } + '401': { schema: AuthError } + '429': { schema: RateLimitError } +``` + +cannot be faithfully represented in the call protocol's `OperationSpec` +today. The adapter would import the `200` output schema and drop the error +schemas — a lossy import that silently discards the operation's failure +contract. + +### Prior art + +The TypeScript reference (`/workspace/@alkdev/operations/src/types.ts` +L38–47, L94, L112) defines `ErrorDefinitionSchema` and an optional +`errorSchemas?: ErrorDefinition[]` on `OperationSpec`: + +```typescript +export const ErrorDefinitionSchema = Type.Object({ + code: Type.String({ description: "Error Code e.g., INVALID_INPUT, NOT_FOUND, UNAUTHORIZED" }), + description: Type.String(), + schema: Type.Unknown(), + httpStatus: Type.Optional(Type.Number()), +}); +``` + +The `mapError()` function (`error.ts` L25–51) matches thrown errors against +the declared error schemas by code prefix — if a handler throws an error +whose message starts with a declared code, `mapError` rewrites it to a +typed `CallError` with that code. This is a proven pattern: operations +declare their error contract, the dispatch machinery maps runtime failures +to the declared codes, and consumers get typed errors instead of string +parsing. + +The translator agent omitted `errorSchemas` from the Rust spec, likely +because it's `Optional` in the TS schema (so dropping it doesn't break the +happy path) and because error schemas are semantically different from +input/output schemas (an operation returns one output but could return any +of several errors). That's a reasonable judgment call for a first +translation pass, but it leaves a real gap for adapters and consumers. + +### The general principle + +This is the same principle as the Safe Exit protocol in the SDD process +(docs/sdd_process.md L19, L423): **make failure a typed, declared thing +rather than an untyped exception that crashes into whoever's listening.** +An operation that declares "I can fail with `FILE_NOT_FOUND`" is the same +shape as an agent that declares "I can fail with `TASK_AMBIGUOUS`" — both +turn an unknown unknown into a known known that the caller can handle +deliberately. + +Complex systems survive not because every component is reliable, but +because failure is expected and typed. Cells have apoptosis (a declared +failure mode that protects the organism). Operations have error schemas (a +declared failure mode that lets the caller handle it). The alternative — +components that fail with untyped strings — is how you get brittle clients +that string-match error messages and break when the message wording +changes. + +## Decision + +### 1. `OperationSpec` gains an optional `error_schemas` field + +```rust +pub struct OperationSpec { + pub name: String, + pub namespace: String, + pub op_type: OperationType, + pub visibility: Visibility, + pub input_schema: Value, + pub output_schema: Value, + pub access_control: AccessControl, + pub error_schemas: Vec, // NEW — empty vec = no declared errors +} + +pub struct ErrorDefinition { + /// Machine-readable error code. e.g., "FILE_NOT_FOUND", "RATE_LIMITED", + /// "INSUFFICIENT_CREDITS". Distinct from the protocol-level codes + /// (NOT_FOUND, FORBIDDEN, etc.) — these are operation-level domain codes. + pub code: String, + + /// Human-readable description of when this error occurs. + pub description: String, + + /// JSON Schema for the error detail payload. The `call.error` event's + /// `details` field conforms to this schema when this error code is + /// returned. `Value` (serde_json::Value) carrying a JSON Schema, same + /// as input_schema/output_schema. + pub schema: Value, + + /// HTTP status code for adapter projection. `from_openapi` maps OpenAPI + /// response status codes to error definitions; `to_openapi` projects + /// error definitions back to response status codes. Optional — not all + /// error sources are HTTP-backed. + pub http_status: Option, +} +``` + +`error_schemas` is a `Vec`, not `Option>`. An +empty vec means "this operation declares no specific domain errors" (it may +still fail with protocol-level codes like `INTERNAL`). This avoids the +`None` vs `Some([])` ambiguity and matches the TypeScript reference's +optional-array convention. + +### 2. The `call.error` payload gains an optional `details` field + +```json +{ + "code": "FILE_NOT_FOUND", + "message": "file not found: /etc/nonexistent", + "retryable": false, + "details": { "path": "/etc/nonexistent", "errno": 2 } +} +``` + +- `code` — the error code. Either a protocol-level code (`NOT_FOUND`, + `FORBIDDEN`, `INVALID_INPUT`, `INVALID_OPERATION_TYPE`, `INTERNAL`, + `TIMEOUT`) or an operation-level domain code from `error_schemas` (e.g., + `FILE_NOT_FOUND`, `RATE_LIMITED`). +- `message` — human-readable error message. Unstructured — for logging and + debugging, not for programmatic handling. Consumers should switch on + `code`, not parse `message`. +- `retryable` — whether the caller should retry. `true` for transient + failures (`TIMEOUT`, `RATE_LIMITED`), `false` for permanent ones + (`NOT_FOUND`, `FORBIDDEN`, `FILE_NOT_FOUND`). +- `details` — optional. When the error code matches a declared + `ErrorDefinition`, `details` conforms to that definition's `schema`. When + the error is protocol-level (`NOT_FOUND`, `FORBIDDEN`, etc.), `details` + is absent or carries protocol-specific context (e.g., the operation name + for `NOT_FOUND`). This field is the typed error payload — it's what + makes errors structured instead of string-matched. + +### 3. Protocol-level vs operation-level error codes + +The six existing codes are **protocol-level** — emitted by the dispatch +machinery, not by handlers: + +| Code | Emitted by | Meaning | +|------|-----------|---------| +| `NOT_FOUND` | Registry | Operation not registered (or Internal op called from wire) | +| `FORBIDDEN` | Registry / ACL | Caller lacks required scopes, or unauthenticated | +| `INVALID_INPUT` | Registry | Input doesn't match `input_schema` | +| `INVALID_OPERATION_TYPE` | Registry / `OperationEnv` | Wrong dispatch path for the operation's type (`invoke()` on a `Sub`, `invoke_streaming()` on a `Query`/`Mutation`, `invoke_sink()` on a `Query`/`Mutation`/`Sub`, or `OperationEnv::invoke()` on a `Sub` during composition — alkcall ADR-021; `Pub` ops dispatch via `invoke_sink()`, alkcall ADR-046) | +| `INTERNAL` | Registry / Adapter | Handler panic, unhandled error, connection failure | +| `TIMEOUT` | Adapter | Request timed out | + +Operation-level domain codes are emitted by **handlers** — the operation's +own logic determines what went wrong. They are declared in `error_schemas` +and appear in the `code` field of `call.error`. Examples: `FILE_NOT_FOUND`, +`PERMISSION_DENIED`, `RATE_LIMITED`, `INSUFFICIENT_CREDITS`, +`CONTEXT_OVERFLOW`. + +The two namespaces are distinct but share the `code` field. Consumers +should handle protocol-level codes uniformly (they mean the same thing +regardless of operation) and operation-level codes per-operation (they +mean what the operation's `error_schemas` says they mean). Unknown codes +— whether a future protocol code or an undeclared operation code — should +be treated as `INTERNAL` with `retryable: false` (same as the current +guidance in the alkcall crate's `call-protocol.md`). + +### 4. Handler error mapping + +When a handler returns an error, the dispatch machinery maps it to a +`call.error` event. The mapping: + +1. If the handler returns a structured error with a `code` that matches a + declared `ErrorDefinition.code`, the `call.error` carries that code and + the error's detail payload (validated against the definition's `schema`). +2. If the handler returns a structured error with a `code` that doesn't + match any declared `ErrorDefinition`, the `call.error` carries + `INTERNAL` with the original code in `details`. This is an undeclared + error — the handler returned a typed error but didn't declare it. +3. If the handler returns an unstructured error (a string, a generic + `Error`, a panic), the `call.error` carries `INTERNAL` with + `retryable: false`. This is the current behavior for all handler + errors. + +The TypeScript `mapError()` function (error.ts L25–51) implements case 2 +and 3 by matching error messages against declared codes. The Rust +implementation can use a typed error return from the handler (`Result` where `CallError` carries a `code`), which is cleaner than +message-string matching — the handler returns a typed error, the registry +checks whether the code is declared, and the `call.error` is constructed +accordingly. + +### 5. `from_openapi` and `to_openapi` error fidelity + +`from_openapi` maps OpenAPI response status codes to `ErrorDefinition`s: + +```rust +// OpenAPI: 404: { schema: NotFoundError } +// → ErrorDefinition { code: "HTTP_404", http_status: Some(404), schema: NotFoundError } +``` + +**Normative rule (review #002 W20)**: `from_openapi` must not produce error +codes that collide with the six protocol-level codes (`NOT_FOUND`, +`FORBIDDEN`, `INVALID_INPUT`, `INVALID_OPERATION_TYPE`, `INTERNAL`, +`TIMEOUT`). The adapter prefixes +imported error codes with `HTTP_` and the status number (e.g., `HTTP_404`, +`HTTP_429`) to avoid collision. This is a requirement for the adapter, not +a naming convention — the `from_openapi` example above was previously shown +producing `NOT_FOUND` from a 404, which collided with the protocol-level +`NOT_FOUND` (operation not registered). The `details` field disambiguates +in practice (present for operation-level, absent for protocol-level), but +this ADR says "consumers should switch on `code`, not parse `message`" — so +the `code` alone must be unambiguous. Operations that hand-write their own +`ErrorDefinition`s should use domain-specific codes (`FILE_NOT_FOUND`, +`RATE_LIMITED`) rather than reusing protocol codes. + +The adapter maps the OpenAPI error schema to the call protocol's JSON Schema +format (same conversion as input/output schemas). The `http_status` field +records the original status code so `to_openapi` can project it back. + +`to_openapi` projects `error_schemas` back to OpenAPI response definitions: + +```yaml +responses: + '200': { schema: } + '404': { schema: } # where http_status = 404 + '429': { schema: } # where http_status = 429 +``` + +This makes the adapter contract from +[ADR-017](017-call-protocol-client-and-adapter-contract.md) faithful on the +error axis — no silent dropping of error contracts. Both `from_openapi` and +`to_openapi` are implemented in alkhttp; the `HTTP_` mapping is part +of this crate's gateway error-fidelity contract (ADR-047). + +`from_mcp` and `to_mcp` follow the same pattern: MCP tool definitions carry +error descriptions, and the adapters map them to/from `ErrorDefinition`s. + +### 6. `services/schema` exposes error schemas + +`services/schema` returns the full `OperationSpec` including `error_schemas`. +A consumer querying `/services/schema` for `/fs/readFile` gets: + +```json +{ + "name": "fs/readFile", + "namespace": "fs", + "op_type": "query", + "input_schema": { ... }, + "output_schema": { ... }, + "error_schemas": [ + { "code": "FILE_NOT_FOUND", "description": "The file does not exist", + "schema": { "type": "object", "properties": { "path": { "type": "string" } } }, + "http_status": null }, + { "code": "PERMISSION_DENIED", "description": "OS-level read permission denied", + "schema": { "type": "object", "properties": { "path": { "type": "string" }, "errno": { "type": "integer" } } }, + "http_status": null } + ] +} +``` + +This enables client code generation: a TypeScript or Rust client generator +reading the schema can produce a typed `Result` +enum instead of a generic `Result`. + +## Consequences + +**Positive:** + +- Operations declare their failure modes. Consumers get typed errors instead + of string-matched messages. This is the same type-safety property that + `input_schema` and `output_schema` provide, extended to the error axis. +- `from_openapi` and `to_openapi` are faithful on the error axis. An + OpenAPI operation's error contract is no longer silently dropped on + import or absent on export. The adapter contract from ADR-017 is now + complete. +- Client code generation can produce typed error enums. A consumer calling + `/fs/readFile` can match on `FILE_NOT_FOUND` vs `PERMISSION_DENIED` + instead of parsing `message` strings. +- The protocol-level vs operation-level distinction is explicit. Protocol + codes (`NOT_FOUND`, `FORBIDDEN`, etc.) mean the same thing regardless of + operation. Operation codes (`FILE_NOT_FOUND`, `RATE_LIMITED`) mean what + the operation declares. No conflation. +- The `details` field carries structured error context that conforms to a + schema — the error payload is typed, not a bare string. This enables + programmatic error handling (retry logic, user-facing error messages, + logging) without string parsing. +- The principle generalizes: making failure a typed, declared thing is the + same pattern as the SDD process's Safe Exit protocol (typed agent + failure) and the same pattern complex biological systems use (apoptosis + as a declared cell failure mode). The more components declare their + failure modes, the more robust the system. + +**Negative:** + +- `OperationSpec` gains a field. Operations that don't declare errors + (empty `error_schemas` vec) still work — the field is additive. But + operations that *should* declare errors and don't will produce `INTERNAL` + with `retryable: false`, same as today. The gap is visible but not + enforced — an operation can ship without error schemas and consumers get + untyped errors for it. This is a documentation/guidance issue, not a + type-system issue. +- The `call.error` payload gains a `details` field. This is a wire-format + addition. Existing clients that only read `code` and `message` are + unaffected (they ignore `details`). New clients can read `details` for + structured error context. This is backward-compatible — `details` is + optional and absent for protocol-level errors. +- Handler error mapping adds a step to the dispatch path: the registry + checks whether the handler's error code matches a declared + `ErrorDefinition`. This is a `HashMap` lookup by code — negligible cost. +- The `http_status` field on `ErrorDefinition` is HTTP-specific. Operations + that aren't HTTP-backed (local, session, from_mcp) leave it as `None`. + This is a pragmatic choice: `from_openapi`/`to_openapi` need it, and it's + optional for everything else. A future non-HTTP adapter that needs a + different error projection field would add it — but `http_status` covers + the immediate use case. +- The TypeScript `mapError()` uses message-string matching to map thrown + errors to codes. The Rust implementation can do better (typed `CallError` + return from handlers), but this means the `Handler` type's return is + `Result` rather than `Result>`. + This is a cleaner API but a slight constraint on handler authors — they + return typed errors, not generic ones. Mitigated: `CallError::internal()` + is available for errors that don't fit a declared code. + +## Assumptions + +1. **Operations can enumerate their meaningful failure modes at + registration time.** If an operation has failure modes that are only + discoverable at runtime (e.g., a dynamic API that returns novel error + codes), those would be `INTERNAL` with `details` carrying the upstream + error. The assumption is that most operations have a knowable set of + domain errors. + +2. **Error codes are stable per operation.** Once an operation declares + `FILE_NOT_FOUND`, consumers depend on that code. Changing it (renaming to + `NOT_FOUND_FILE`) is a breaking change for consumers that match on it. + This is the same stability property as `input_schema` and + `output_schema` — the operation's interface is its contract. Adding new + error codes is additive (consumers that don't know the new code treat it + as `INTERNAL`); removing or renaming codes is breaking. + +3. **Protocol-level codes are distinct from operation-level codes.** If an + operation declares a code that collides with a protocol code (e.g., an + operation declares `NOT_FOUND` as a domain error), the protocol code + takes precedence in the dispatch machinery (the registry's `NOT_FOUND` + for "operation not registered" is emitted before the handler runs). The + assumption is that operations use domain-specific codes (`FILE_NOT_FOUND`) + rather than reusing protocol codes (`NOT_FOUND`). This is a naming + convention, not a type-system enforcement. + +4. **`details` is optional and backward-compatible.** Existing clients that + ignore `details` continue to work. New clients read `details` for + structured context. The wire format addition is additive. + +## References + +- [ADR-017](017-call-protocol-client-and-adapter-contract.md): Call protocol + client and adapter contract (adapter fidelity — + this ADR makes `from_openapi`/`to_openapi` faithful on the error axis; the + decision record is alkcall ADR-022) +- [ADR-014](014-secret-material-flow-and-capability-injection.md): Secret + material flow (the `details` field must not carry secret + material — same constraint as `metadata`) +- [ADR-015](015-privilege-model-and-authority-context.md): Privilege model + (the `FORBIDDEN` protocol code covers ACL + denial; operation-level `PERMISSION_DENIED` is a distinct domain error + for OS-level permission issues) +- alknet mono-repo `docs/reviews/001-pre-implementation-architecture-sanity-check.md` + (finding C5, which this ADR resolves) +- alkcall ADR-021: Streaming handler for subscriptions (alknet ADR-049, + ported here as [ADR-049](049-streaming-handler-for-subscriptions.md); + amends this ADR's protocol-level code list — `INVALID_OPERATION_TYPE` + added as the sixth protocol-level code) +- alkcall ADR-046: `Pub` operation type and `HandlerKind::Sink` — the + producer→consumer streaming primitive; `Pub` ops dispatch via + `invoke_sink()` +- alknet mono-repo `docs/sdd_process.md` L19, L423 (Safe Exit protocol — the + general principle of making failure typed and declared) +- TypeScript reference: `/workspace/@alkdev/operations/src/types.ts` + L38–47 (`ErrorDefinitionSchema`), L94, L112 (`errorSchemas` on + `OperationSpec`), `error.ts` L25–51 (`mapError`) + +## Port notes + +- Renames: "alknet-http" → alkhttp; "alknet-core"/"alknet-call" → alkcall + ("the call crate (now alkcall)"). +- `OperationType::Subscription` → `Sub` (alkcall rename; alkcall ADR-046 + added `OperationType::Pub`, producer→consumer streaming via + `call.published`, `HandlerKind::Sink`). Corrections flowing from this: + the `INVALID_OPERATION_TYPE` table row now lists the dispatch-path + mismatches per the current handler-kind model (`invoke()` on a `Sub`, + `invoke_streaming()` on a `Query`/`Mutation`, `invoke_sink()` on a + `Query`/`Mutation`/`Sub`, and `OperationEnv::invoke()` on a `Sub` during + composition) and cites alkcall ADR-021 (this crate's + [ADR-049](049-streaming-handler-for-subscriptions.md)) and alkcall ADR-046 + for `Pub`/`Sink`. `from_openapi` produces no `Pub` ops in v1 (SSE + responses detect as `Sub`), so the adapter-fidelity discussion is + unaffected by `Pub`. +- Producer/consumer terminology: "A client calling `/fs/readFile`" → "a + consumer"; "clients get typed errors" → "consumers get typed errors"; + §2 `code` bullet "Clients should switch on `code`" → "Consumers should + switch on `code`"; §3 "Clients should handle protocol-level codes" → + "Consumers should handle". HTTP/MCP/OpenAPI client references + (§5's "HTTP clients", the client code generation, "brittle clients") + keep their names — those are inherent-directionality roles of the + external systems, not call-protocol roles. Wire-format backward + compatibility bullets retain "existing clients" wording (wire consumers). +- `from_openapi` example: the original said the adapter maps "the OpenAPI + error schema to alknet's JSON Schema format" — corrected to "the call + protocol's JSON Schema format". +- §5 closing sentences extended to note that `from_openapi`/`to_openapi` + live in alkhttp and that the `HTTP_` rule is part of this crate's + gateway error-fidelity contract (ADR-047) — port framing, not a new + decision. +- Cross-reference remappings (verified alknet→alkcall ADR mapping): alknet + ADR-017 (adapter contract) → alkcall ADR-022; alknet ADR-049 (streaming + handler) → alkcall ADR-021. ADR-014 and ADR-015 are ported to this crate + under the same numbers and linked. +- Review/spec paths (`docs/reviews/...`, `docs/sdd_process.md`, + `call-protocol.md` L-references) are alknet mono-repo artifacts; annotated + as alknet-record citations. The `call-protocol.md` relative link and its + line-number citation for the unknown-code rule became a textual "the + alkcall crate's `call-protocol.md`" reference. +- No decision content changed — the `error_schemas` field, the `details` + payload field, the protocol-vs-operation code namespace split, the handler + error mapping, the `HTTP_` prefix rule, and the `services/schema` + exposure are verbatim from the alknet ADR modulo the corrections logged + above. \ No newline at end of file diff --git a/docs/architecture/decisions/027-tls-identity-redesign-acme-rawkey-decoupling.md b/docs/architecture/decisions/027-tls-identity-redesign-acme-rawkey-decoupling.md new file mode 100644 index 0000000..264b98d --- /dev/null +++ b/docs/architecture/decisions/027-tls-identity-redesign-acme-rawkey-decoupling.md @@ -0,0 +1,374 @@ +# ADR-027: TLS Identity Redesign — ACME Integration + RawKey Decoupling + +*Ported from alknet ADR-027 (TLS Identity Redesign — ACME Integration + RawKey Decoupling); re-targeted to alkhttp.* + +## Status + +Accepted (§5 amended by alknet ADR-083 — the `acme-tls/1` guard moves from +`dispatch_quinn` to the shared `dispatch` method, since ACME challenges +arrive over TCP+TLS, not QUIC; the rationale holds, only the location +changes) + +## Context + +> **Port note (scope):** This ADR is ported because it is the decision of +> record for the browser-facing TLS constraint this crate inherits: +> **browsers require X.509** — they cannot present or verify RFC 7250 raw +> Ed25519 keys, so the browser-reachable surface of alkhttp (`http/1.1`, +> `h2`, and the WebSocket upgrade path) must be served from an X.509 +> identity (CA-issued via ACME, or operator-provided). The TLS machinery +> itself — `TlsIdentity`, `TlsSetup`, ACME provisioning, the rustls +> server config — is **not implemented in this crate**; alkhttp is +> transport-coupling-free by design (alknet ADR-027's alknet-internal +> sections describe the alknet endpoint layer, which remains an alknet +> concern; see §Port notes at the end of this file). The clauses below +> that matter to alkhttp are the browser constraint and the identity +> modes it implies; the alknet-internal provisioning mechanics are +> retained for provenance and marked as alknet concerns. + +OQ-12 marked "resolved" identified two TLS identity use cases: RFC 7250 +raw Ed25519 keys (default, P2P) and X.509 certs (domain-hosted, browsers). +ACME auto-provisioning was described as "additive — it will be adapted +when domain-hosted nodes need it." That deferral created two +architectural issues that surface now that ACME is a concrete target. + +### Issue 1: `TlsIdentity` cannot represent ACME *(alknet concern)* + +`TlsIdentity` is `#[derive(Debug, Clone)]` and lives in `StaticConfig` — +a static, synchronous config value. ACME requires: + +- A long-lived async state machine (`AcmeState` event loop, spawned for + the endpoint's lifetime) that handles ordering, challenge response, + cert renewal, and cache I/O. +- TLS-ALPN-01 challenge handling: `acme-tls/1` must be in the server's + `alpn_protocols`, and a `ResolvesServerCertAcme` must serve challenge + certs during the TLS handshake. +- Config fields: domains, cache directory, ACME directory URL, contact + email. + +`AcmeState` is not `Clone`. It cannot be a `TlsIdentity` variant. The +current `build_rustls_server_config(&TlsIdentity) -> ServerConfig` is +synchronous — there's no room for spawning an async state machine or +holding a runtime resolver handle. The reverse-proxy project solved this +with a two-phase construction: static config → `TlsMode` (runtime +objects) → `ServerConfig`. alknet needs the same split. + +*(This issue, and the two-phase construction that resolves it, live in +the alknet endpoint layer — not in alkhttp, which owns no TLS config.)* + +### Issue 2: `RawKey` is coupled to the `iroh` feature *(alknet concern)* + +`TlsIdentity::RawKey(iroh::SecretKey)` is gated `#[cfg(feature = "iroh")]`. +The `RawKeyCertResolver` and `Ed25519SigningKey` impls are gated +`#[cfg(all(feature = "quinn", feature = "iroh"))]`. This means a +quinn-only build (the default feature set) **cannot use RFC 7250 raw-key +identity** — the very mode described as "default for most alknet nodes." + +The coupling is artificial. `iroh::SecretKey` is a thin newtype over +`ed25519_dalek::SigningKey` (`pub struct SecretKey(SigningKey)`). The +alknet code uses exactly three APIs: `.public().as_bytes()`, `.sign(msg)`, +and `.clone()`. None of these are iroh-specific. The raw-key TLS path +needs Ed25519 signing + SPKI encoding — both available from +`ed25519-dalek` + `rustls` without iroh. + +The iroh *transport* (`build_iroh_endpoint`) does need `iroh::SecretKey` +for `iroh::Endpoint::builder().secret_key(...)`. If `TlsIdentity::RawKey` +no longer carries an `iroh::SecretKey`, the iroh transport must convert +from the new key type — trivial since `iroh::SecretKey::from_bytes(&[u8; +32])` accepts raw Ed25519 key bytes. + +*(This issue is entirely within the alknet transport/config layer; alkhttp +has no `TlsIdentity` and no iroh dependency.)* + +### ACME challenge handling with quinn (QUIC, not TCP) *(alknet concern)* + +Research confirmed how TLS-ALPN-01 works with quinn: + +- The `ResolvesServerCertAcme` resolver intercepts the challenge at the + **cert resolution step**, during the TLS handshake, before the + handshake result is surfaced to the application. +- When an ACME CA connects with ALPN `[acme-tls/1]`, rustls calls the + resolver, which returns the challenge cert. The handshake completes. + The CA inspects the cert's SAN and validates the challenge — no + application-layer data exchange needed. +- quinn's `connecting.await` then returns a completed `Connection` with + ALPN `acme-tls/1`. alknet's `dispatch_quinn` would find no handler for + that ALPN and close the connection. **The challenge already succeeded** + — the close is cosmetic. +- Unlike the reverse-proxy (TCP + `LazyConfigAcceptor`), quinn gives no + "peek at ClientHello" hook. The challenge is fully TLS-layer-handled; + the application only needs to close challenge connections gracefully + (silent close, not a "no handler" warning). + +Key constraint: ACME requires `with_cert_resolver(ResolvesServerCertAcme)`, +not `with_single_cert`. You cannot just append `acme-tls/1` to an +`X509`/`SelfSigned` config — there'd be no resolver to serve the +challenge cert. ACME is a distinct `ServerConfig` construction path. + +*(Challenge handling at this layer is an alknet endpoint concern.)* + +## Decision + +### 1. Add `TlsIdentity::Acme` variant (static config data only) *(alknet concern)* + +```rust +pub enum TlsIdentity { + X509 { cert: PathBuf, key: PathBuf }, + RawKey(Ed25519SecretKey), // see Decision 3 + SelfSigned, + Acme { // NEW + domains: Vec, + cache_dir: PathBuf, + directory: AcmeDirectory, // enum: Production, Staging, Custom(String) + contact: Vec, // e.g. ["mailto:admin@example.com"] + }, +} +``` + +`Acme` holds only static, `Clone`/`Debug`-safe config data. No +`AcmeState`, no resolver, no runtime objects. The async state machine is +constructed at endpoint setup time (Decision 2). + +### 2. Split server-config construction into two phases *(alknet concern)* + +Replace the synchronous `build_rustls_server_config(&TlsIdentity) -> +ServerConfig` with a two-phase construction: + +**Phase 1 — `TlsSetup` (async, at endpoint construction):** + +```rust +struct TlsSetup { + server_config: rustls::ServerConfig, + acme_state: Option, // spawned task + handle for shutdown +} +``` + +For `X509`, `SelfSigned`, `RawKey`: construct `ServerConfig` +synchronously (current path, unchanged). `acme_state` is `None`. + +For `Acme`: construct `AcmeConfig`, spawn the `AcmeState` event loop, +get `ResolvesServerCertAcme`, build `ServerConfig` with +`with_cert_resolver(resolver)`, add `acme-tls/1` to `alpn_protocols`. +`acme_state` is `Some(handle)` so the endpoint can abort the ACME task +on shutdown. + +**Phase 2 — use `TlsSetup.server_config` to build `quinn::ServerConfig`:** + +Same as today: `QuicServerConfig::try_from(rustls_config)` → +`quinn::ServerConfig::with_crypto(...)`. + +The `TlsSetup` is constructed inside `AlknetEndpoint::new()` (or +`run_quinn_accept_loop`), not inside `TlsIdentity`. The `TlsIdentity` +enum stays a pure data structure. + +### 3. Decouple `RawKey` from iroh — use `ed25519-dalek` directly *(alknet concern)* + +Replace `TlsIdentity::RawKey(iroh::SecretKey)` with +`TlsIdentity::RawKey(Ed25519SecretKey)`, where `Ed25519SecretKey` is a +thin alknet-core-owned wrapper over `ed25519_dalek::SigningKey`: + +```rust +pub struct Ed25519SecretKey(ed25519_dalek::SigningKey); +``` + +This type is `Clone`, `Debug` (redacting), `Zeroize`, and not gated +behind any feature flag. `ed25519-dalek` becomes a direct dependency of +alknet-core (it's already in the dependency tree transitively via iroh). + +The `RawKeyCertResolver` and `Ed25519SigningKey` rustls impls move from +`#[cfg(all(feature = "quinn", feature = "iroh"))]` to +`#[cfg(feature = "quinn")]` — raw-key TLS identity works in quinn-only +builds. + +The `iroh` feature gate on `TlsIdentity::RawKey` is removed. The +variant is always available. + +### 4. iroh transport converts from `Ed25519SecretKey` *(alknet concern)* + +`build_iroh_endpoint` currently reads `TlsIdentity::RawKey(iroh::SecretKey)` +and passes it to `iroh::Endpoint::builder().secret_key(...)`. After +decoupling, it converts: + +```rust +if let Some(TlsIdentity::RawKey(key)) = static_config.tls_identity.as_ref() { + let iroh_key = iroh::SecretKey::from_bytes(key.as_bytes()); + builder = builder.secret_key(iroh_key); +} +``` + +`iroh::SecretKey::from_bytes(&[u8; 32])` accepts raw Ed25519 key bytes — +no information loss. This conversion is `#[cfg(feature = "iroh")]` only. + +### 5. ACME ALPN challenge handling in `dispatch` (moved from `dispatch_quinn` by alknet ADR-083) *(alknet concern)* + +Add an early-return guard in `dispatch` (the shared dispatch path, +moved from `dispatch_quinn` by alknet ADR-083) before the handler lookup: + +```rust +// In the shared `dispatch` method (moved from `dispatch_quinn` by alknet ADR-083): +if alpn == b"acme-tls/1" { + debug!("acme-tls/1 challenge connection completed at TLS layer; closing"); + connection.close(0u32.into(), b"acme done"); + return; +} +``` + +This avoids the misleading "no handler for ALPN" warning. The challenge +is already answered at the TLS layer; the application just closes +gracefully. No `ProtocolHandler` registration for `acme-tls/1`. The +guard is transport-agnostic — it fires for any connection whose TLS +handshake negotiated `acme-tls/1`, regardless of which transport +delivered it. In practice ACME TLS-ALPN-01 challenges arrive over +TCP+TLS (CAs validate via TCP to port 443, not QUIC); advertising +`acme-tls/1` on a QUIC listener that shares the ACME config is +harmless. See alknet ADR-083 for the full rationale. + +*(The guard lives in the alknet endpoint's ALPN dispatch loop. alkhttp's +`HttpAdapter` — the `ProtocolHandler` for `h2`/`http/1.1` — never sees +`acme-tls/1` connections; TLS identity and ALPN dispatch remain alknet +concerns per alknet ADR-010/ADR-001.)* + +### 6. Feature-gate ACME behind a new `acme` feature *(alknet concern)* + +Add a `acme` feature to alknet-core: + +```toml +[features] +acme = ["dep:rustls-acme"] +``` + +`TlsIdentity::Acme` is available regardless of feature (it's just config +data), but constructing `TlsSetup` with an `Acme` variant requires the +`acme` feature. Without it, `TlsIdentity::Acme` at endpoint construction +returns an error ("ACME feature not enabled"). This keeps the +footprint down for nodes that don't need ACME — `rustls-acme` and its +dependencies are only compiled when the feature is on. + +### 7. `acme-tls/1` in ALPN list only when ACME is active *(alknet concern)* + +When `TlsIdentity::Acme` is configured, `acme-tls/1` is appended to the +`alpn_protocols` list alongside the handler ALPNs. When ACME is not +configured, `acme-tls/1` is not advertised — no behavior change for +non-ACME nodes. + +## What alkhttp inherits from this decision + +- **Browsers require X.509.** Browsers cannot verify or present RFC 7250 + raw Ed25519 keys; any deployment that serves browsers (the WebSocket + browser bidirectional path — alkhttp ADR-044/ADR-048 — and the gateway + endpoints) needs the hub's TLS listener to present an X.509 + certificate chain (WebPKI/CA-issued, e.g. via ACME) rather than a raw + key. This is the constraint that motivates the ACME work above, and it + is why alknet ADR-027 is ported here at all. +- **Identity selection is upstream of alkhttp.** The `HttpAdapter` + registers on the standard HTTP ALPNs (`h2`, `http/1.1`) per alkhttp + ADR-001/ADR-002 and is transport-agnostic: the TLS identity that + secures the listener (raw key for P2P, X.509 for browser-facing) is + chosen by the consumer's assembly/endpoint layer, not by this crate. + +## Consequences + +- **Breaking change to `TlsIdentity`** *(alknet concern)*: `RawKey(iroh::SecretKey)` → + `RawKey(Ed25519SecretKey)`. Pre-1.0 crate, in-repo consumers only. + The assembly layer and tests that construct `TlsIdentity::RawKey` must + update. +- **`ed25519-dalek` becomes a direct dependency** of alknet-core *(alknet concern)*. It's + already in the dependency tree (transitive via iroh), so no new + compilation cost for `iroh` builds. Quinn-only builds that were not + using `RawKey` before will now compile `ed25519-dalek` — it's a small, + pure-Rust crate with no C dependencies. +- **`rustls-acme` is feature-gated** (`acme` feature) *(alknet concern)*. Nodes not using + ACME don't compile it. The feature is compatible with `quinn` (ACME + is quinn-only; iroh uses its own TLS). +- **`build_rustls_server_config` becomes async** (or is replaced by an + async `TlsSetup::new`) *(alknet concern)*. The accept loop already runs in an async + context, so this is a local change. +- **ACME state machine lifecycle** *(alknet concern)*: the `AcmeState` task is spawned in + `AlknetEndpoint::new()` and aborted on shutdown. The `TlsSetup` struct + carries the `JoinHandle` so `AlknetEndpoint::shutdown()` can abort it. +- **No handler needed for `acme-tls/1`** *(alknet concern)*: the `dispatch_quinn` guard + handles it. `HandlerRegistry` is not involved. +- **For alkhttp**: no API surface change. The crate gains a documented + constraint — browser-facing deployments require X.509 on the listener — + which its consumers must satisfy at the endpoint layer. + +## Alternatives Considered + +### A. ACME as a `ResolvesServerCert` wrapper behind `X509` *(alknet concern)* + +OQ-12 suggested ACME "fits naturally as an additional `TlsIdentity` +variant or as a `rustls::ResolvesServerCert` implementation behind the +existing `X509` path." The second option — wrapping `X509` — was +rejected because ACME needs async state + config fields (domains, cache, +contact) that don't fit behind the static `X509 { cert, key }` variant. +A `ResolvesServerCert` that internally does ACME would need to be +constructed at config time with those fields, which means `X509` would +need to carry them — bloating the variant for non-ACME users. A +dedicated `Acme` variant is cleaner. + +### B. Keep `RawKey` coupled to iroh, only add ACME *(alknet concern)* + +Rejected because the coupling is the root cause of quinn-only builds not +supporting the "default" identity mode. Fixing only ACME would leave the +artificial iroh dependency in place. Since both changes touch +`TlsIdentity` and `build_rustls_server_config`, doing them together +avoids two breaking changes to the same enum. + +### C. Use `iroh::SecretKey` for both, re-export from alknet-core *(alknet concern)* + +Rejected because it would make `iroh` a non-optional dependency of +alknet-core, defeating the feature-gated transport design (alknet +ADR-010). `ed25519-dalek` is a lightweight, pure-Rust crate; `iroh` is +not. + +### D. Register a no-op `ProtocolHandler` for `acme-tls/1` *(alknet concern)* + +Rejected because it would require the handler registry to know about +ACME (a TLS-layer concern), polluting the ALPN dispatch abstraction. +The `dispatch_quinn` guard is a one-line check that keeps ACME handling +in the endpoint layer where it belongs. + +## Cross-References + +- OQ-12 (TLS identity provisioning) — updated by this ADR *(alknet OQ)* +- alknet ADR-010 — multi-connectivity endpoint, feature-gated + transports; the ALPN router and endpoint that owns TLS identity and + dispatch (an alknet decision; not ported to alkhttp — see Port notes) +- alknet ADR-004 — auth as shared core (an alknet decision; ported to + alkhttp as alkhttp ADR-004, same number, different scope) +- `docs/architecture/crates/core/endpoint.md` *(alknet doc)* — TLS identity use cases +- `docs/architecture/crates/core/config.md` *(alknet doc)* — `TlsIdentity` enum +- `/workspace/@alkdev/reverse-proxy/src/tls/` — proven ACME implementation pattern +- `rustls-acme` crate — ACME state machine + cert resolver +- alkhttp ADR-044 — the WebSocket browser path that makes the X.509 + requirement load-bearing for this crate +- alkhttp ADR-034 — browsers are not peers; the public X.509 endpoint + role and the hub role in the peer model + +## Port notes + +- All TLS provisioning mechanics (`TlsIdentity`, `TlsSetup`, + `build_rustls_server_config`, `AcmeState`, the `acme` feature, the + `acme-tls/1` dispatch guard) are **alknet concerns** — they live in the + alknet endpoint/config layer (alknet ADR-010, ADR-001), not in + alkhttp. alkhttp is transport-coupling-free (no TLS, no endpoint, no + accept loop); sections describing them are marked *"(alknet + concern)"* inline and retained for provenance. The clause that is + substantive for alkhttp — **browsers require X.509 for browser-facing + TLS** — is restated under "What alkhttp inherits from this decision." +- `alknet-core` ownership of `Ed25519SecretKey` is historical: the core + types now live in the alkcall crate (vendored there). The decision + text is preserved as written; the wrapper type is an alkcall-owned + type today. +- Reference links rewritten per alkhttp docs conventions: the original + relative sibling-ADR links to alknet ADR-010 and alknet ADR-004 are + textual references above (neither is ported to alkhttp under those + numbers/slug; alkhttp ADR-004 is a different document — auth as + shared core). The `crates/core/*.md` spec links are dropped in + favor of the textual "alknet doc" annotations because the alknet spec + tree is not part of this crate's docs. +- The original ADR-083 is an alknet decision (shared `dispatch` guard + relocation); it is cited textually as "alknet ADR-083" and is not + ported. +- Original title preserved: "TLS Identity Redesign — ACME Integration + + RawKey Decoupling". \ No newline at end of file diff --git a/docs/architecture/decisions/034-outgoing-only-x509-and-three-peer-roles.md b/docs/architecture/decisions/034-outgoing-only-x509-and-three-peer-roles.md new file mode 100644 index 0000000..2042386 --- /dev/null +++ b/docs/architecture/decisions/034-outgoing-only-x509-and-three-peer-roles.md @@ -0,0 +1,541 @@ +# ADR-034: Outgoing-Only X.509 and the Three Peer Roles + +*Ported from alknet ADR-034 (Outgoing-Only X.509 and the Three Peer Roles); re-targeted to alkhttp.* + +## Status + +Accepted (resolves OQ-37) + +> **Port note (emphasis):** This port emphasizes **§4 — browsers are not +> peers** — because that clause governs alkhttp's browser-facing surface +> (the WebSocket path, alkhttp ADR-044/ADR-048). The X.509/TLS machinery +> this ADR discusses (server cert verifiers, `TlsIdentity`, WebPKI +> verification, fingerprint pinning) is **an alknet concern**: alkhttp is +> transport-coupling-free and owns no TLS configuration or verifier +> selection. The X.509-relevant sections (§2, §3, §5) are retained for +> provenance and marked accordingly; what alkhttp consumes is the peer- +> model closure of §4 and the three-role vocabulary of §1. + +## Context + +OQ-37 framed the open question as: "the three credential types (Ed25519, +X.509, bearer token) and how X.509 server identity fits the peer model." +During resolution, it became clear that **three distinct remote roles** +had been conflated under the single label "X.509 endpoint," and that the +conflation was the actual source of the confusion — not the TLS +mechanics, which alknet ADR-027 and alknet ADR-030 had already settled. + +The three roles are real and structurally different: + +1. **Public X.509 endpoint** — a remote HTTPS or `alk/call`-over-TLS + server reachable by domain name, authenticated by a CA-issued X.509 + cert. The local node is a *client* of it. Examples: a + third-party API (`vast.ai`, `api.openai.com`), a public hub + that the local node dials over the open internet, an `alk/call` + peer that has chosen to expose a domain + X.509 instead of (or in + addition to) an Ed25519 raw key. The client authenticates to the + server by **bearer token** (browsers and most HTTP clients cannot do + TLS client-auth); the server authenticates to the client by **CA + verification** (WebPKI), not by fingerprint pinning. + +2. **Transport relay** — iroh's DERP-equivalent (`iroh-relay`). A + connectivity-assistance node that forwards encrypted datagrams + between peers who cannot directly connect (NAT traversal). It is + *infrastructure*, not an application peer: it does not + register operations, does not participate in the call protocol's + peer graph, and has no `PeerEntry` / `PeerId` in the auth + model. Nodes inherit it for free when the `iroh` feature is on; the + relay's own identity (an Ed25519 `NodeId`) is iroh's concern, not + the protocol stack's. *(Transport-relay mechanics are an alknet + concern; alkhttp has no iroh dependency and no relay role — the + relay is named here only to keep the three-role vocabulary intact.)* + +3. **Hub / hosting node** — an application peer that acts as a + hub in a hub-and-spoke (head/worker) topology. It is an ordinary + `PeerEntry` that *happens* to also expose a public domain + X.509 + (so browsers / external HTTPS clients can reach it) *and* an Ed25519 + identity (so other nodes can reach it P2P via iroh or direct + quinn). The git-hosting-relay-with-gossip-sync use case is this role: + the hub is a full peer that additionally serves browsers. *(The hub's + browser-facing surface is what alkhttp serves — the gateway endpoints + and the WebSocket session.)* + +The pre-ADR-034 framing asked whether `PeerEntry` should be made +**symmetric** — i.e., whether the local node should hold a `PeerEntry` +for *every* remote it might dial, including pure-public-API servers it +has no P2P relationship with. This ADR answers **no**: the asymmetry is +correct and reflects a real difference in trust model. `PeerEntry` (and +the `PeerId` it produces) is the model for **peers in the call-protocol +peer graph** (alkcall ADR-024) — peers that get a stable logical +identity, are addressable via `PeerRef::Specific`, and whose ops land in +the peer-keyed overlay. A pure-client connection to a public HTTPS API is +not that. + +This distinction matters because forcing a stable logical `peer_id` +onto "the operator of `api.example.com`" is wrong: a public domain's +operator can change hands, the cert can be reissued, and the local node +has no stable logical identity to attach — only "domain X verified by +CA Y today." That is a different trust model from "this Ed25519 key is +`worker-a`, and key rotation updates the fingerprint but not the +identity" (alkcall ADR-025). + +## Decision + +### 1. Name the three roles; stop using "relay" ambiguously + +The architecture documents use three distinct terms: + +| Role | Identity | Transport | Peer? | Example | +|------|----------|-----------|--------------|---------| +| **Public X.509 endpoint** | Domain + CA-issued X.509 | HTTPS / `alk/call`-over-TLS | No (client only, unless also role 3) | `api.alk.dev`, `vast.ai` | +| **Transport relay** | iroh `NodeId` (Ed25519) | iroh's DERP-like protocol | No (infrastructure) | `relay.iroh.network` | +| **Hub / hosting node** | Ed25519 raw key **and/or** X.509 | iroh / direct quinn / HTTPS | Yes (full `PeerEntry`) | git-hosting hub, head node | + +Existing specs that say "relay" when they mean "domain-hosted service" +or "hub" are amended by reference to this table. alknet ADR-027's "domain- +hosted services" and alkcall ADR-025's "X.509 cert" credential path refer +to the **public X.509 endpoint** role and the **hub** role; iroh's +transport relay is a separate, inherited component referenced only in +the iroh transport path *(alknet concern)*. + +### 2. Outgoing-only X.509 is not a `PeerEntry` on the client side *(TLS mechanics: alknet/alkcall concern; the peer-model rule: alkhttp-relevant)* + +When a `CallClient` (or the alkhttp `from_openapi` / `from_mcp` adapters) +dials a remote that is a **public X.509 endpoint** and the local node has +no P2P relationship with it (no `PeerEntry` for the remote): + +- The server is authenticated by **CA verification** + (`rustls::WebPkiServerVerifier` with the platform root store or a + configured CA bundle) *(the verifier itself is alknet/alkcall dial-layer + machinery — alkhttp consumes its outcome, it does not build + verifiers)*. There is no fingerprint to pin — pinning a + `SHA256:` fingerprint against an external CA-issued cert + is brittle (cert renewal changes the fingerprint) and is not the + WebPKI trust model. The trigger for CA verification is **the absence + of a `PeerEntry` for the remote combined with an X.509 transport**; + the verifier selection rule is stated in full in §3 below. The + `ConnectionCredentials.remote_identity: Option` field + (alkcall ADR-012, extending alkcall ADR-022 §7) carries an expected + fingerprint/cert when the caller has one to pin (`Some`); for a + pure-client X.509 dial with no `PeerEntry`, `remote_identity` is + `None` and the CA path applies. The `Option` is load-bearing — `None` + is the public-X.509-endpoint state, not a missing field: an + implementer must not default it to a placeholder, and must not treat + `None` as "skip verification" (`None` + X.509 = CA verification; + `None` + Ed25519 raw key = fail closed). (alkcall ADR-022 §7 specified + `remote_identity` as "expected fingerprint or cert"; this ADR extends + its semantics so that `remote_identity: None` + no `PeerEntry` + + X.509 transport selects CA verification, and `remote_identity: None` + + Ed25519 raw-key transport fails closed.) +- The client authenticates to the server by **bearer token** + (`ConnectionCredentials.auth_token`), carried in the call-protocol + `auth_token` payload field (or the HTTP `Authorization` header for + alkhttp's `from_openapi` / `from_mcp`). What the *server* does with + that token depends on which kind of public X.509 endpoint it is: + - **Third-party API** (`api.openai.com`, `vast.ai` — not an alknet + node): the server applies its own auth scheme (its own API-key + validation, its own ACL). The protocol's `PeerEntry` / `ApiKeyEntry` + types do not apply on the far side; the client just carries the + token in the shape the remote expects (an HTTP header, a + call-protocol `auth_token` payload) and treats the remote's + response as authoritative. + - **Hub reached over its public X.509 path** (a role-3 hub + dialed over the domain instead of P2P): the hub resolves the + client's token via its own `PeerEntry.auth_token_hash` or + `ApiKeyEntry` — the *server's* bookkeeping, not the client's. The + client still holds no `PeerEntry` for the hub on its own side + unless it also has a P2P trust relationship with that hub (in which + case the §3 mixed-fingerprint path applies, not this one). +- The client may still present its TLS client cert (Ed25519 raw public + key, per alknet OQ-29) when one is configured; bearer token is the + *authorization* credential, and TLS client-auth (when presented) is + *additional* identity material the server may use. For a third-party + API the cert is ignored; for a hub it may be extracted as a + fingerprint. Presenting or omitting the client cert is the caller's + choice via `ConnectionCredentials`; this ADR does not require + disabling client-auth on this path. *(TLS client-cert presentation is + dial-layer machinery — an alknet/alkcall concern, not alkhttp's.)* +- The connection does **not** get a `PeerId` on the client side. It is + not added to `PeerCompositeEnv` (alkcall ADR-024). There is no + `PeerRef::Specific` routing to it. The connection is a live + `CallConnection` (or, for alkhttp's reqwest-backed client host, an + HTTP client session) the caller holds directly; ops discovered via + the `from_call`-pattern import (alkcall ADR-028) or via alkhttp's + `from_openapi` / `from_mcp` land in that connection's Layer 2 overlay + (alkcall ADR-019) and are invoked through the connection handle, not + through the peer-keyed routing layer. + +This is the **asymmetry** OQ-37 worried about, stated as a deliberate +design property: `PeerEntry` is for peers in the call-protocol peer +graph. Pure-client connections to public X.509 endpoints are not in +that graph on the client side. The server may have a `PeerEntry` for +*us* (resolving our bearer token, in the hub sub-case); we +don't need one for *it*. + +### 3. The hub case is already covered by ADR-030's mixed-fingerprint `PeerEntry` *(alknet/alkcall concern)* + +A **hub / hosting node** that is reachable both P2P (Ed25519 raw key +via iroh or direct quinn) and via a public domain (X.509 for browsers) +is a single `PeerEntry` with mixed fingerprints: + +```rust +PeerEntry { + peer_id: "hub-a".into(), + fingerprints: vec![ + "ed25519:", // P2P path + "SHA256:", // HTTPS / browser-facing path + ], + auth_token_hash: Some(""), + scopes: vec![...], + resources: {...}, + ... +} +``` + +*(The `PeerEntry` struct with mixed fingerprints is an alkcall type — +alkcall ADR-025. Fingerprint normalization across quinn/iroh — +alkcall ADR-025 §6 — is an alknet/alkcall dial-layer concern.)* + +When a node dials this hub P2P, the Ed25519 fingerprint +matches; when it dials over the public X.509 path (e.g., because P2P +connectivity failed), the X.509 fingerprint matches — both resolve to +the same `peer_id` (`"hub-a"`). The X.509 path here uses +**fingerprint pinning** (the `SHA256:` is in `PeerEntry`), *not* +CA verification, because the local node has a prior P2P trust +relationship with this specific hub and has recorded its cert's +fingerprint. This is the one case where X.509 fingerprint pinning is +correct: the peer is a known peer, not an arbitrary public API. + +The choice between **CA verification** (role 1) and **fingerprint +pinning** (role 3, X.509 path) is driven by whether the local node has +a `PeerEntry` for the remote — this is the authoritative verifier +selection rule, referenced from §2: + +| Local has `PeerEntry` for remote? | Remote cert type | Client verifier | +|----------------------------------|------------------|-----------------| +| No (public X.509 endpoint) | X.509 | `WebPkiServerVerifier` (CA verification) | +| No | Ed25519 raw key | fails closed (no CA to fall back to — raw-key remotes are always known peers; fingerprint IS identity) | +| Yes (hub, Ed25519 path) | Ed25519 raw key | fingerprint match (`ed25519:`) | +| Yes (hub, X.509 path) | X.509 | fingerprint match (`SHA256:`) | + +This is the key-type-aware verifier from alknet OQ-29, with the +*peer-model* criterion made explicit: the verifier choice is determined +by whether the remote is a known peer (`PeerEntry` present → pin) or an +external server (`PeerEntry` absent → CA, or fail closed for raw keys). +*(The verifier construction itself is dial-layer machinery — an +alknet/alkcall concern; alkhttp's `from_openapi`/`from_mcp`/client host +consume connections established under this rule but do not select +verifiers.)* + +### 4. Browsers connecting to a hub are not peers + +A browser reaching a hub over WebTransport (or HTTPS — and, per alkhttp +ADR-044, over WebSocket) is served by the hub's HTTP handler — in the +extracted crate tree, **alkhttp**. The browser authenticates by **bearer +token** (HTTP `Authorization`), resolved by the hub's +`IdentityProvider::resolve_from_token` against the hub's +`PeerEntry.auth_token_hash` or `ApiKeyEntry`. The browser is **not a +peer on the hub's side either** — it does not get a `PeerId`, does +not enter `PeerCompositeEnv`, and its "ops" are HTTP routes / browser +streams served by alkhttp, not entries in the call-protocol +peer-keyed overlay. The hub's `PeerEntry` for the browser (if any) is +about authorizing the bearer token, not about peer-graph membership. + +This keeps the peer graph populated only by full nodes (role 3 +hubs and role-3-style spoke nodes), never by browsers or pure HTTP +clients. + +> **Amendment (rationale added by alknet ADR-044 §5, ported to alkhttp +> ADR-044):** The closure above is correct but states the conclusion +> without the supporting argument. The distinction that makes it correct +> is: **"peer" means an addressable node in the call-protocol peer +> graph** — a stable `PeerId`, reachable via `PeerRef::Specific`, whose +> ops land in `PeerCompositeEnv`, whose identity is stable across +> reconnects. It does *not* mean "any endpoint that exchanges calls +> during a live session." A browser is the second thing but not the +> first, on three concrete grounds: (1) no stable cryptographic identity +> of its own (it presents a bearer token the hub issued; nothing to +> pin), (2) ephemeral (close the tab → connection dies → the +> connection-local overlay dies with it; a `PeerEntry` keyed to a +> browser would be dead within seconds), (3) not addressable from other +> nodes (another node has no way to reach "the browser currently +> connected to hub-A"; the hub holds it as a live `CallConnection` +> handle, not a peer-graph entry). The connection-local Layer 2 overlay +> (alkcall ADR-019; the inbound mirror of §2 above) is what gives the +> browser bidirectional-call capability *without* peer-graph membership. +> This rationale is transport-agnostic — it applies to WebSocket (the +> browser path in alkhttp, alkhttp ADR-044) and to WebTransport +> (an alknet transport; not in alkhttp scope — see alkhttp ADR-069) +> equally. See alkhttp ADR-044 §5 for the full statement. + +### 5. WebTransport relay-as-proxy is a transport-only feature, scoped separately *(alknet concern)* + +> **Port note:** WebTransport is **not in alkhttp scope at all** — per +> alkhttp ADR-069 it was removed from this crate entirely (it is an +> alknet concern). This section is retained for provenance of the +> *auth-model* point (the proxy is transport-only and does not change +> identity resolution), which remains true regardless of where the +> proxy lives. + +A **WebTransport proxy** that terminates the browser's WebTransport +connection and proxies encrypted traffic to a hub's P2P endpoint +(avoiding the need for the hub itself to expose a public X.509 endpoint) +is a real feature, especially for the browser-to-P2P-peer case. It is +**not** load-bearing on the auth model resolved here: + +- The proxy does not change how identities resolve. The browser still + authenticates by bearer token; the hub still resolves it via + `PeerEntry.auth_token_hash`. The proxy is transport-only. +- The fingerprint normalization committed in alkcall ADR-025 §6 + (`ed25519:` for raw keys across quinn and iroh) was already + designed to keep the proxied path clean: a proxied connection's + Ed25519 identity is the same `ed25519:` whether the client + connected directly or through the proxy. + +> **Amendment (wording only — the decision stands):** An earlier draft +> of this section framed the relay-as-proxy as belonging to an +> "h3/WebTransport deferral bucket" and "lands when `h3` / +> WebTransport lands." That framing was a residual of the "two-way door +> as deferral" anti-pattern (alkcall ADR-032 §"What this framework is +> NOT") that alknet ADR-038 was later written to reject. alknet ADR-038 +> has since been **superseded by alknet ADR-044**, which re-defers +> `h3`/WebTransport as a genuine scope decision (the browser +> bidirectional path uses WebSocket). In the alkhttp crate, the scope +> question is closed differently: **WebTransport is removed from alkhttp +> entirely** (alkhttp ADR-069) — there is no "revives later" posture in +> this crate. The *auth-model* decision in this §5 (the proxy is +> transport-only; it does not change identity resolution) is unchanged +> by any of these ADRs. The *scope* question (which crate hosts the +> proxy, if it is ever built) remains an alknet concern — the alknet +> OQ-38 tracking is not carried into alkhttp. + +### 6. On-chain / smart-contract peer discovery fits the OQ-36 adapter pattern *(alknet/alkcall concern)* + +The downstream use case — storing relay/repo info and org/user ACL on a +smart-contract platform, with relays (hubs) syncing git repos via +iroh's gossip protocol — is a **discovery and ACL-source** concern, not +an auth-model concern. It does not change any of decisions 1–4: + +- The hubs are role-3 `PeerEntry` peers (mixed fingerprints, full peer- + graph membership, gossip-synced). +- The smart contract is a **source of `PeerEntry` records**. It maps + cleanly onto the repo/adapter pattern (alknet ADR-033): a future + on-chain peer-store adapter implementing `IdentityProvider` + against a smart contract is additive, exactly like a SQLite peer + store (alknet ADR-035). The auth model (`PeerEntry`, `PeerId`, + `Identity`) is unchanged; only the *source* of the records changes. +- The repo/ACL data on-chain is consumed by the hub's authorization + layer (`AccessControl::check` against scopes/resources populated from + the on-chain `PeerEntry`), not by the TLS / fingerprint path. + +Designing that adapter now would be premature — it is downstream of +both the repo/adapter exploration (alknet OQ-36) and the git crate +(alknet OQ-10). It is noted here only to confirm it does not reopen +OQ-37. *(This is recorded for provenance; the peer-store adapters are +alknet/alkcall assembly-layer components, not alkhttp surface.)* + +## What this does NOT change + +- **`PeerEntry` struct shape** (alkcall ADR-025) — unchanged. Mixed + fingerprints (Ed25519 + X.509) were already supported. +- **`Identity` / `IdentityProvider` trait** — unchanged (vendored in the + alkcall crate). The verifier choice is a `CallClient` / + `from_openapi` / `from_mcp` dial-layer concern, not an + `IdentityProvider` concern. +- **`ConnectionCredentials` struct** — unchanged (alkcall ADR-012). + `remote_identity` already carries the expected key type; this ADR + specifies how the verifier is chosen from it (CA for unknown X.509 + remotes, fingerprint match for known peers). +- **`PeerCompositeEnv` / `PeerRef`** (alkcall ADR-024) — unchanged. + Pure-client X.509 connections simply do not enter the peer-keyed + overlay. +- **`TlsIdentity`** (alknet ADR-027) — unchanged. The server-side X.509 / ACME + / RawKey modes are unaffected; this ADR is about the *client-side* + verifier choice for outgoing connections. *(And `TlsIdentity` itself + is an alknet concern — not present in alkhttp.)* +- **The no-env-vars invariant** — unaffected. The bearer token for the + outgoing X.509 case still comes from `Capabilities` (alkcall ADR-010 + secret-material flow), not env vars. + +## Consequences + +**Positive:** +- OQ-37 is resolved. The "make `PeerEntry` symmetric" instinct is + rejected with a clear criterion: `PeerEntry` is for peers in the + call-protocol peer graph; pure-client connections to public X.509 + endpoints are not in that graph on the client side. +- The three remote roles are named, so future specs and conversations + can distinguish "public X.509 endpoint," "transport relay," and + "hub / hosting node" instead of overloading "relay." +- The client-side verifier choice has a single rule: known peer + (`PeerEntry` present) → fingerprint pin; unknown X.509 remote + (`PeerEntry` absent) → CA verification. This closes the + `AcceptAnyServerCertVerifier` security hole for X.509 that alknet + OQ-29 flagged, with the peer-model criterion made explicit. +- The hub case (mixed Ed25519 + X.509 fingerprints, browser access via + HTTPS and the browser transports) is confirmed to need no new types — + alkcall ADR-025's `fingerprints: Vec` already covers it. +- The relay-as-proxy and on-chain-discovery use cases are + recorded with clear homes (the relay-as-proxy is a transport-only + feature whose scope is an alknet concern — not tracked in alkhttp; + the on-chain discovery follows the alknet OQ-36 adapter pattern) so + they don't get lost and don't reopen the auth model. + +**Negative:** +- The dial-layer client paths (alkcall's `CallClient`; alkhttp's + `from_openapi` / `from_mcp` outbound adapters via the reqwest-backed + client host) must respect the "is this remote a known `PeerEntry`?" + distinction when a TLS client config is built for an outgoing + connection. In alkhttp the credential injection point is the adapter + layer (alkhttp ADR-014 secret-material flow): `ConnectionCredentials` + — including `remote_identity` — are supplied by the consumer's + assembly layer, and the verifier itself is built in the dial layer + (alknet/alkcall), not in this crate. This is a small implementation + cost and is local to connection establishment; it is not a structural + change. +- Operators must understand the distinction between "I have a + `PeerEntry` for this remote (pin its fingerprint)" and "I'm calling a + public API (trust the CA)." In practice this is intuitive (it's the + difference between `~/.ssh/known_hosts` and a browser's CA trust + store), but the docs must state it clearly, which this ADR and the + spec amendments do. +- Pure-client X.509 connections have no `PeerId` on the client side, so + any future feature that wants to route to "the connection I opened to + `api.alk.dev`" must hold the `CallConnection` handle directly rather + than using `PeerRef::Specific`. This is the correct constraint — + `PeerRef::Specific` is for known peers, not for arbitrary dials — but + it is a constraint downstream code must respect. + +## Assumptions + +1. **A remote reachable by Ed25519 raw key is always a known peer.** + Raw-key remotes have no CA; the fingerprint IS the trust anchor. An + unknown Ed25519 remote cannot be verified at all (there is no CA to + fall back to), so the connection fails closed. This means the + "public X.509 endpoint" role is the *only* role where the local node + dials a remote it has no `PeerEntry` for. This is correct and + intended — it is the same model iroh uses. *(The iroh transport is an + alknet concern; the fail-closed rule is stated here because it is a + property of the credential model, not of any transport.)* + +2. **Browsers never enter the peer-keyed overlay.** A browser is + served by alkhttp (gateway routes and, per alkhttp ADR-044/ADR-048, + the WebSocket session) and authenticates by bearer token. The hub may + have a `PeerEntry` for the browser's token (to authorize it), but the + browser is not a `PeerId`-bearing peer. This is the explicit closure + of the "browser as peer" path — browsers are clients, not peers. + **The rationale** (addressability vs. bidirectionality — a browser + has no stable identity of its own, is ephemeral, and is not + addressable from other nodes) is stated in alkhttp ADR-044 §5, which + amends §4 above by reference. The closure applies transport- + agnostically. + +3. **X.509 fingerprint pinning is only for known hubs.** Pinning an + X.509 fingerprint for an arbitrary public API is brittle (cert + renewal) and is not done. The `PeerEntry.fingerprints` X.509 entry + is for the hub case where the local node has a P2P trust + relationship and wants to also recognize the hub's domain-facing + cert. + +4. **The on-chain / smart-contract discovery use case does not change + the auth model.** It is a source of `PeerEntry` records, implemented + as an additive `IdentityProvider` adapter (alknet ADR-033 / alknet + OQ-36). The hub-and-gossip topology it implies is built from role-3 + hubs, which this ADR confirms are ordinary `PeerEntry` peers. + +## References + +- OQ-37 (resolved by this ADR) — the three auth types and how X.509 + server identity fits the peer model +- alknet [ADR-027](027-tls-identity-redesign-acme-rawkey-decoupling.md) — + `TlsIdentity` (RawKey / X509 / Acme), the browser limitation (no RFC + 7250), WebTransport requires X.509 *(ported to alkhttp — same file + name in this directory; the TLS machinery itself is an alknet + concern)* +- alkcall ADR-024 — the peer-keyed overlay model that `PeerEntry` / + `PeerId` feed into; pure-client connections are not in this graph +- alkcall ADR-025 — `PeerEntry` with mixed fingerprints; fingerprint + normalization (`ed25519:` across quinn/iroh); the `SHA256:` X.509 + fingerprint format +- alknet ADR-033 / alknet ADR-035 — the repo/adapter pattern that an + on-chain `IdentityProvider` adapter follows and the concrete SQLite + adapter shape (the on-chain adapter would follow the same trait + + separate-crate pattern) *(alknet decisions, not ported to alkhttp)* +- alkcall ADR-022 §7 — `CallCredentials` (now `ConnectionCredentials`, + alkcall ADR-012) with `remote_identity`; alkcall ADR-022 specified + "expected fingerprint or cert", this ADR §2 extends its semantics so + that `remote_identity: None` + no `PeerEntry` + X.509 transport + selects CA verification +- alkcall ADR-019 — the Layer 2 per-connection overlay where ops + discovered via the `from_call` pattern (alkcall ADR-028) or via + alkhttp's `from_openapi` / `from_mcp` on a pure-client X.509 + connection land +- alknet OQ-29 (resolved) — key-type-aware server cert verification; + this ADR adds the peer-model criterion (known peer vs. public X.509 + endpoint) that selects the verifier +- alknet OQ-10 (deferred) — git adapter scope; the on-chain / + gossip-synced git-hosting hub use case in §6 is downstream of the git + crate *(alknet OQ)* +- alknet OQ-36 (resolved by alknet ADR-035) — concrete persistence + adapter shapes; the on-chain `IdentityProvider` adapter in §6 follows + the same repo/adapter pattern *(alknet OQ)* +- alknet ADR-038 — **superseded by alknet ADR-044**; not ported to + alkhttp. See alknet ADR-034 §5's amendment history and alkhttp + ADR-069 (WebTransport removed from alkhttp scope). +- alkhttp ADR-044 — the WebSocket browser path; §5 states the + "browser is not a peer" rationale that amends this ADR's §4 +- alkhttp ADR-048 — the WebSocket session shape; the browser bidirectional + path this ADR's §4 governs +- iroh transport relay (`iroh-relay`) — referenced to distinguish it + from the hub role *(iroh docs; an alknet transport concern)* +- alkcall crate docs — `CallClient`/`ConnectionCredentials` dial path, + verifier selection by `PeerEntry` presence; see the alkcall crate's + own documentation for the client-and-adapters spec + +## Port notes + +- The call protocol core types are vendored in the **alkcall** crate + (old alknet-core + alknet-call merged). All type-level references are + re-cited to alkcall ADRs: peer-keyed overlay/`PeerCompositeEnv` is + alkcall ADR-024, `PeerEntry`/fingerprint normalization is alkcall + ADR-025, `CallClient`/adapter contract is alkcall ADR-022 (§7 + credentials), `ConnectionCredentials` is alkcall ADR-012, Layer 2 + registry layering is alkcall ADR-019, `from_call` as a manual free + function is alkcall ADR-028. The alknet ADR numbers that originally + carried these (ADR-029, ADR-030, ADR-017, ADR-024) are alknet numbers + and do not coincide with alkcall or alkhttp numbers; each citation + above names the owning crate explicitly. +- `CallCredentials` → `ConnectionCredentials`: alkcall ADR-012 renamed + the credential struct when it decoupled dial from call; the ported + text uses the current name with the original alkcall ADR-022 §7 + citation preserved. +- `alknet/call` (the old ALPN string) → `alk/call` (alkcall ADR-004 + `alk/` convention). Table and prose updated. +- "alknet-http" → "alkhttp"; "alknet node"/"alknet peer" phrasing + generalized to "node"/"peer" where the sentence is about the protocol + model rather than the alknet binary. +- §4 heading and prose originally said "over WebTransport"; the port + adds "(or HTTPS — and, per alkhttp ADR-044, over WebSocket)" and + attributes the served surface to alkhttp. WebTransport references are + marked as alknet concerns; in alkhttp, WebTransport is out of scope + entirely (alkhttp ADR-069), not deferred. +- §2, §3, §5, and §6 are retained for provenance but annotated: TLS + verifier selection, fingerprint pinning, the iroh transport relay, and + the on-chain peer-store adapter are dial-layer/alknet concerns, not + alkhttp surface. §4 is the load-bearing clause for this crate and is + the emphasis of this port. +- alknet OQ references (OQ-29, OQ-36, OQ-37, OQ-10, OQ-38) are alknet + open-questions records, not ported into alkhttp's OQ file; they are + cited textually as "alknet OQ-NN" (alkcall ADR-024 is cited for the + peer-graph model rather than alknet OQ-37's original resolution + context). +- Reference links rewritten: `../../decisions/...` and + `../../crates/...` relative links replaced per alkhttp docs + conventions; links into the old call/core spec trees became textual + "alkcall crate docs" references. alknet ADR-027 is linked as a + sibling file because it is ported to alkhttp (same number and slug). +- The original also cited `docs/research/alknet-http/phase-0-findings.md` + (DH-2) and iroh reference docs; these are alknet research artifacts + and are referenced textually only. +- Original title preserved: "Outgoing-Only X.509 and the Three Peer + Roles". \ No newline at end of file diff --git a/docs/architecture/decisions/036-http-to-call-operation-mapping.md b/docs/architecture/decisions/036-http-to-call-operation-mapping.md new file mode 100644 index 0000000..7884116 --- /dev/null +++ b/docs/architecture/decisions/036-http-to-call-operation-mapping.md @@ -0,0 +1,303 @@ +# ADR-036: HTTP-to-Call Operation Mapping + +*Ported from alknet ADR-036 (HTTP-to-Call Operation Mapping); re-targeted to alkhttp.* + +## Status + +Proposed — **routing decision superseded by +[ADR-047](047-remove-direct-call-http-surface.md)** (the direct-call +surface `POST /{service}/{op}` is removed; the gateway `/call` is the +sole invoke path). ADR-036's other clauses — SSE projection, Bearer +auth, `/healthz`, stealth decoy, error mapping, `External`-only +dispatch — remain in force (see ADR-047 §"What survives from +ADR-036"). The `to_openapi` clause was already superseded by ADR-042. + +## Context + +`alkhttp` implements `ProtocolHandler` for the standard HTTP ALPNs (`h2`, +`http/1.1`; `h3`/WebTransport is deferred per +[ADR-044](044-defer-webtransport-browsers-use-websocket.md)). An inbound +HTTP request that targets an alkhttp operation +must become a call-protocol `call.requested` dispatch — the HTTP handler is a +*projection* of the call protocol, not a parallel routing layer. The +question is how an HTTP request maps to an operation invocation. + +Three options were considered in the alkhttp Phase 0 research +(alknet mono-repo: `docs/research/alknet-http/phase-0-findings.md`, +decision point DH-3): + +- **(a) Direct path mapping.** `POST /{service}/{op}` → `call.requested` for + `/{service}/{op}`. The HTTP handler parses the request body as the + operation input, sends `call.requested`, and returns the response as JSON. + The HTTP surface is a thin projection of the call protocol's + `/{service}/{op}` operation path format (resolved by alknet OQ-13). +- **(b) OpenAPI-defined routes.** The HTTP surface is defined by the + `to_openapi` projection — routes, methods, schemas are generated from the + registry's `External` operations, and the HTTP handler dispatches based on + the generated OpenAPI spec's path mapping. +- **(c) Explicit route registration.** The assembly layer registers HTTP + routes explicitly, mapping URL paths to operations. Most flexible, most + boilerplate. + +This is a load-bearing architectural choice. Once the HTTP surface's routing +contract is published and external clients build against it, changing the +mapping (e.g., from "the HTTP path IS the operation path" to "the HTTP path +is a generated alias") is a one-way door: every client breaks. It needs an +ADR before implementation. + +The call protocol's operation path format is `/{service}/{op}` (alknet +OQ-13, resolved). The HTTP handler serves these operations over HTTP. The +mapping must be a *projection* of that single operation surface, not a second +routing table that has to be kept in sync with the registry. + +## Decision + +> **Routing decision superseded by +> [ADR-047](047-remove-direct-call-http-surface.md).** The direct-call +> surface defined below (`POST /{service}/{op}` → `call.requested`) is +> removed — the gateway's `/call` endpoint ([ADR-042](042-openapi-gateway-pattern.md)) +> is the sole invoke +> path over HTTP. This section is retained as the historical record of +> the original decision; ADR-047 records the reversal and what survives. +> The `to_openapi` clause below was already superseded by ADR-042 (see +> the amendment in this section). + +**Direct path mapping is the default HTTP surface; `to_openapi` is the +discovery/projection layer, not a parallel router.** + +The `HttpAdapter` receives an HTTP request whose path is `/{service}/{op}` +(e.g., `POST /fs/readFile`, `POST /agent/chat`), constructs a +`call.requested` dispatch with `operationId: /{service}/{op}` and `input: +`, and returns the operation's response as JSON. The HTTP path +IS the operation path — one routing surface, the call protocol's. + +`to_openapi` generates the OpenAPI spec that *describes* this surface for +external consumers (route paths, methods, request/response schemas, error +schemas per [ADR-023](023-operation-error-schemas.md)). It does not define +separate routes — the generated +spec's `paths` mirror the `/{service}/{op}` operation paths. An external +client reading the OpenAPI doc learns the same routes the HTTP handler +serves; there is no second mapping. + +> **Amendment (superseded by [ADR-042](042-openapi-gateway-pattern.md) on +> the `to_openapi` clause):** The paragraph above described the original +> "per-operation-paths projection" — `to_openapi` generating one OpenAPI +> path entry per `External` operation, mirroring `/{service}/{op}`. ADR-042 +> replaces this with the **gateway pattern**: `to_openapi` generates 5 +> fixed gateway endpoints (`/search`, `/schema`, `/call`, `/batch`, +> `/subscribe`) instead of one path per operation (the 5-endpoint set, +> extended with `/publish` in alkhttp — ADR-068). The "no second routing +> table" property is preserved (the gateway endpoints are fixed; the +> per-caller operation surface is discovered via `/search`, not preloaded +> into a generated path set). The direct-call surface (`POST +> /{service}/{op}`) that this ADR defines was **unchanged at the time** +> — ADR-042 only changed what `to_openapi` *describes*, not what the +> HTTP handler *serves*. **The direct-call surface was later removed by +> [ADR-047](047-remove-direct-call-http-surface.md)** (the gateway +> `/call` is the sole invoke path; the simplified contract is a few +> fixed endpoints, not a per-operation REST tree). A traditional +> per-operation-paths OpenAPI projection remains available as an +> additive alternative (ADR-042 §5), and a deployment that wants the +> former direct-call HTTP surface builds it as a custom route +> projection ([ADR-046](046-assembly-layer-custom-http-routes.md)). + +### HTTP method semantics + +The call protocol's `OperationType` (`Query`, `Mutation`, `Sub`, +per the alkcall crate's `docs/architecture/operation-registry.md`) maps to +HTTP methods on the default surface: + +| `OperationType` | Default HTTP method | Notes | +|-----------------|----------------------|-------| +| `Query` | `GET` | Read-only, idempotent. Input from query parameters + optional body. | +| `Mutation` | `POST` (or `PUT`/`PATCH`/`DELETE` if the operation declares it) | Default `POST`; the op may declare a specific mutation method in its spec metadata. | +| `Sub` | `GET` with `Accept: text/event-stream` | Streaming — the HTTP handler projects the subscription's `call.responded` stream as SSE chunks. | + +The default method for an `External` operation with no explicit HTTP method +declared is `POST` for `Mutation`, `GET` for `Query`. This is the +least-surprise default; an operation that wants a specific HTTP verb +declares it. The method-to-`OperationType` mapping is a two-way-door +default (changing it later is additive — a new method is added, existing +methods keep working). + +### Streaming projection (SSE) + +A `Sub` operation served over HTTP/1.1 or HTTP/2 projects its +`call.responded` stream as Server-Sent Events. Each `call.responded` event +becomes an SSE `data:` frame; `call.completed` closes the SSE stream; +`call.aborted` closes the stream with an SSE error event. This is the +HTTP/1.1 + HTTP/2 streaming projection. Over WebSocket (the v1 browser +bidirectional path, ADR-044), the subscription projects directly onto the +WS connection — `call.responded` events as binary WS messages, no SSE +framing. WebTransport (`h3`) would project onto WebTransport bidirectional +streams but is deferred per ADR-044. + +### Auth + +Inbound HTTP auth is `Authorization: Bearer `, resolved via +`IdentityProvider::resolve_from_token()` (the alknet mono-repo `auth.md` +handler table — `HttpAdapter`, Bearer header, `resolve_from_token`). This +is settled by [ADR-004](004-auth-as-shared-core.md) and alknet OQ-11; this +ADR does not change it. Bearer-only is the auth +mechanism; other HTTP auth schemes (Basic, API key in query param) are not +implemented. An unauthenticated request to an operation with +`AccessControl` restrictions returns `401`/`403` (mapped from the call +protocol's `FORBIDDEN` protocol code). + +### Stealth mode + +The HTTP handler on `h2`/`http/1.1` serves a decoy (configurable: fake +404, a static site, a redirect) for paths that are not registered +operations. This is the ALPN-based stealth mapping (alknet mono-repo +`endpoint.md`; in alkhttp, [ADR-010](010-alpn-router-and-endpoint.md) and +this crate's `server` spec) — +clients that don't offer the call-protocol ALPNs get the HTTP handler, and +unknown HTTP paths get the decoy. The decoy is a two-way-door config +default (an operator picks what to serve); the *existence* of the stealth +path is fixed by ADR-010. + +### `/healthz` and operational endpoints + +`GET /healthz` is a raw HTTP route outside the call protocol — no auth, no +operation registration. It exists for infrastructure (load balancers, +orchestrators). Other operational endpoints (metrics, dashboard) are +call-protocol operations if built (`/metrics/list`, `/dashboard/view`), +not raw HTTP routes. `healthz` is the one exception: it must be callable +without auth before identity is resolvable. + +## Consequences + +**Positive:** +- One routing surface. The HTTP handler does not maintain a second routing + table; it projects the call protocol's `/{service}/{op}` paths directly. + No sync drift between the operation registry and the HTTP routes. +- `to_openapi` is a pure projection (generate a spec that *describes* the + existing surface), not a routing authority. The generated spec is always + consistent with what the handler actually serves because they're the same + paths. +- External HTTP clients (curl, axios, browser `fetch`) can call alkhttp + operations without knowing about the call protocol — the HTTP surface is + a standard REST-like API. +- The abort cascade (alkcall ADR-020) is preserved: an HTTP client + disconnecting + mid-subscription is detected as a stream close, and the HTTP handler + sends `call.aborted` for the in-flight subscription, which cascades to + descendants. +- The HTTP method mapping (`Query`→`GET`, `Mutation`→`POST`, + `Sub`→`SSE`) is the standard REST projection — no surprise + verbs, no exotic method semantics. + +**Negative:** +- The HTTP surface inherits the call protocol's `/{service}/{op}` path + shape. An operation named `fs/readFile` is served at `POST /fs/readFile`, + not at a REST-nested `POST /fs/files/:id/read` or any other + REST-conventional path. Operations that want a REST-nested HTTP path + must declare it in spec metadata (a two-way-door extension); the + default is the operation path verbatim. This is a deliberate + least-surprise-for-alkhttp choice, not a REST-purist choice. +- HTTP request/response semantics don't map cleanly onto every call + protocol operation. A `Query` with a large input has to put the input in + the body (GET-with-body is non-standard). A `Mutation` that is + idempotent doesn't get `PUT` semantics unless it declares them. The + projection is lossy at the edges; operations that need precise HTTP + semantics declare them. +- `to_openapi` is a published compatibility contract ([ADR-017](017-call-protocol-client-and-adapter-contract.md) + Consequences: + once external clients build against the generated spec, the mapping is + one-way). The generated spec's versioning (tied to the registry's + `External` operation set version) must be emitted as a spec marker so + consumers can detect mapping changes. This is alknet OQ-17's + published-artifact concern, applied to the HTTP projection. + +## Assumptions + +1. **The operation path IS the HTTP path.** An operation `fs/readFile` is + served at `/fs/readFile`. There is no separate HTTP path mapping layer. + If a deployment wants different HTTP paths (e.g., a REST-nested + convention), that's a future projection layer, not a change to this + mapping. + +2. **`External` operations are the HTTP surface.** `Internal` operations + (composition-only, [ADR-015](015-privilege-model-and-authority-context.md)) + are not served over HTTP — they return `404` + on the HTTP handler, matching the call protocol's `NOT_FOUND` for wire + calls to Internal ops. The HTTP handler dispatches only `External` + operations. + +3. **HTTP auth is Bearer-only.** The HTTP handler resolves identity from + the `Authorization: Bearer` header via `resolve_from_token`. Basic auth, + API keys in query params, and other HTTP auth schemes are not + implemented. A deployment that needs a different auth scheme adds it as + middleware (two-way door), but the default surface is Bearer-only. + +## References + +- [ADR-004](004-auth-as-shared-core.md) — `IdentityProvider`, Bearer → + `resolve_from_token` (the auth model this ADR uses, unchanged) +- [ADR-010](010-alpn-router-and-endpoint.md) — stealth mode as ALPN + dispatch (the HTTP handler on standard ALPNs serves the decoy) +- [ADR-015](015-privilege-model-and-authority-context.md) — External/Internal + visibility (Internal ops are not served over HTTP; the decision record + is alkcall ADR-017) +- alkcall ADR-020: Abort Cascade for Nested Calls (alknet ADR-016) — abort + cascade (HTTP client disconnect → `call.aborted` → cascade to + descendants) +- [ADR-017](017-call-protocol-client-and-adapter-contract.md) — + `to_openapi` as a projection; published-spec compatibility contract + (the decision record is alkcall ADR-022) +- [ADR-023](023-operation-error-schemas.md) — error schema fidelity in + `from_openapi`/`to_openapi`; HTTP status mapping (the decision record is + alkcall ADR-016) +- [ADR-042](042-openapi-gateway-pattern.md) — supersedes this ADR's + `to_openapi` clause (the per-operation-paths projection is replaced by + the 5-endpoint gateway pattern — extended with `/publish` in alkhttp, + ADR-068; the direct-call surface this ADR defines is unchanged — *at + the time*; ADR-047 later removes it) +- [ADR-047](047-remove-direct-call-http-surface.md) — supersedes this + ADR's routing decision (the direct-call surface is removed; the + gateway `/call` is the sole invoke path). This ADR's non-routing + clauses survive. +- alknet OQ-13 (resolved) — operation path format `/{service}/{op}` +- alknet mono-repo `docs/research/alknet-http/phase-0-findings.md` DH-3 — + the decision this ADR resolves +- `http-server.md` (in this crate's `docs/architecture/`) — the spec that + implements this mapping (alknet original: `crates/http/http-server.md`) + +## Port notes + +- Renames: "alknet-http" → alkhttp; "alknet" (the system/node being + described) → alkhttp where the HTTP-served surface is meant. +- `OperationType::Subscription` → `Sub` (alkcall rename) in the method + table and the SSE section. alkcall ADR-046 later added + `OperationType::Pub` (producer→consumer streaming via `call.published`, + `HandlerKind::Sink`); `Pub` has no row in this ADR's HTTP method table — + its HTTP projection is the `/publish` gateway endpoint (alkhttp + ADR-068). +- **Gateway endpoint count**: the alknet record's "5 fixed gateway + endpoints" statements are preserved verbatim as history; alkhttp extends + the set with `/publish` (alkhttp ADR-068) — marked with an inline + annotation in the amendment blockquote, not silently rewritten. +- Cross-reference remappings (verified alknet→alkcall ADR mapping): alknet + ADR-016 (abort cascade) → alkcall ADR-020. ADR-004/010/015/017/023 are + ported to this crate under the same numbers and linked; their alkcall + record numbers are noted alongside where cited (015→alkcall ADR-017, + 017→alkcall ADR-022, 023→alkcall ADR-016). +- Producer/consumer terminology: the original contained no call-protocol + server/client role framing; "external HTTP clients (curl, axios, browser + `fetch`)" and similar retain their names — those are inherent-directionality + roles of the HTTP/OpenAPI surface, not call-protocol roles. +- Spec-document links converted per alkhttp conventions: + `crates/http/http-server.md` → `http-server.md` in this crate's + `docs/architecture/`; `operation-registry.md` → textual "the alkcall + crate's docs/architecture/operation-registry.md"; `auth.md` and + `endpoint.md` are alknet mono-repo spec-tree documents cited textually + (the auth model is ADR-004, the stealth mapping ADR-010). +- OQ references (OQ-11, OQ-13, OQ-17) are alknet OQ-tracker items, cited + textually; the operation path format is owned by the alkcall crate. +- The `docs/research/alknet-http/phase-0-findings.md` pointer is kept as a + historical alknet mono-repo reference (no alkhttp equivalent exists). +- No decision content changed — the three-option framing, the direct-path + decision (retained as the historical record of the ADR-047-superseded + routing), the method table, SSE projection, auth, stealth decoy, + `/healthz`, consequences, and assumptions are verbatim from the alknet + ADR modulo the adaptations above. \ No newline at end of file diff --git a/docs/architecture/decisions/037-mcp-stdio-transport-exclusion.md b/docs/architecture/decisions/037-mcp-stdio-transport-exclusion.md new file mode 100644 index 0000000..4c45733 --- /dev/null +++ b/docs/architecture/decisions/037-mcp-stdio-transport-exclusion.md @@ -0,0 +1,204 @@ +# ADR-037: MCP Stdio Transport Exclusion + +*Ported from alknet ADR-037 (MCP Stdio Transport Exclusion); re-targeted to alkhttp.* + +## Status + +Proposed + +## Context + +The Model Context Protocol (MCP) defines multiple transports for +communicating between an MCP client and an MCP server. The MCP Rust SDK +(`rmcp` at `/workspace/rust-sdk/`) implements two: + +1. **Streamable HTTP** (`transport-streamable-http-client-reqwest` for + clients, `transport-streamable-http-server` for servers). The client + connects to an HTTP endpoint; the server serves an HTTP endpoint. + Network-isolated, auth-gatable (Bearer token middleware, per the rmcp + `simple_auth_streamhttp.rs` example), and runs under whatever auth/ + identity/capabilities machinery the host applies to HTTP. + +2. **stdio** (`transport-child-process`). The client spawns the MCP server + as a child process and pipes JSON-RPC over its stdin/stdout. This is + the model the MCP spec promotes for "just download an MCP server and + run it locally." + +The alkhttp crate implements `from_mcp` (import remote MCP tools as +call-protocol operations) and `to_mcp` (expose local operations as MCP +tools). Both are feature-gated behind an `mcp` feature (the rmcp +dependency is optional). The question this ADR resolves is which MCP +transports alkhttp supports. + +### The stdio security problem + +MCP stdio transport is `transport-child-process` — the rmcp client calls +`StdioClientTransport { command, args, env, cwd }`, which spawns an +arbitrary executable and pipes JSON-RPC over its stdin/stdout. An MCP +server is an arbitrary program that the MCP client executes with whatever +privileges the client process has. + +The "download untrusted MCP servers and run them via stdio" model is +indistinguishable from `curl | sh` with extra steps: + +- **Arbitrary code execution.** The MCP server is an executable. Running + it is RCE. There is no sandbox — the child process has the full + privileges of the client process (filesystem, network, environment + variables, ability to spawn further processes). +- **No auth boundary.** The MCP protocol messages flow over stdin/stdout; + there is no TLS, no auth token, no identity resolution. The server is + trusted by construction (you spawned it). +- **The "download untrusted MCP server" UX.** The MCP ecosystem's + promoted workflow is: find an MCP server on a registry, install it, + point your client at it. This is the npm-without-the-checksums model, + but the "package" is a process with full local privileges, not a + library that runs in-process. + +alkhttp's security posture is the opposite of this. alknet-vault is +local-only by construction (alknet ADR-025 — the vault crate and its +decision record live in the alknet mono-repo); the no-env-vars invariant +(ADR-014; spec in the alkcall crate's `client-and-adapters.md`) exists +specifically to avoid the "download untrusted code that reads your +secrets" pattern; capabilities are injected by the assembly layer, not +read from the environment a spawned process can inspect. Building stdio +support into alkhttp would import the exact RCE vector the rest of the +architecture is designed to avoid. + +## Decision + +**alkhttp supports only streamable HTTP for MCP. Stdio is not built.** + +The `mcp` feature gate pulls in rmcp with the streamable HTTP transport +features only: + +```toml +[features] +mcp = [ + "dep:rmcp", + # rmcp client transport (for from_mcp) — streamable HTTP only + # rmcp server transport (for to_mcp) — streamable HTTP only +] +``` + +The stdio transport (`transport-child-process`) is explicitly **not** a +dependency and **not** feature-gated. It is not built, not optional, not +"behind a separate feature." alkhttp's `from_mcp` uses rmcp's +`StreamableHttpClientTransport` (reqwest-based); `to_mcp` uses rmcp's +`StreamableHttpService` (axum-based, a tower service that nests into an +axum `Router` — see the rmcp `simple_auth_streamhttp.rs:134-159` +example). No stdio code path exists in the crate. + +### If someone wants stdio MCP + +They run it themselves, outside alkhttp. An operator who wants to use a +stdio-only MCP server can spawn it as a subprocess, run a small +streamable-HTTP-to-stdio bridge, and point `from_mcp` at the bridge's +HTTP endpoint. That bridge is the operator's responsibility — alkhttp +does not ship it, does not endorse it, and the bridge is where the RCE +risk lives, explicitly in the operator's hands, not hidden behind an +alkhttp feature flag. + +This is the same posture as alknet ADR-025 (vault local-only dispatch: +remote vault access requires a separate crate with its own ADR and +threat model): the dangerous thing is not built by default; if someone +wants it, they build it themselves and own the security model. + +## Consequences + +**Positive:** +- alkhttp does not import the MCP stdio RCE vector. There is no code + path in alkhttp that spawns an arbitrary executable. +- The streamable HTTP path is network-isolated, auth-gatable (Bearer + middleware), and runs under alkhttp's auth/identity/capabilities + machinery — the same machinery that gates every other HTTP request. +- `from_mcp` operations (imported MCP tools) are `Internal` by default + ([ADR-015](015-privilege-model-and-authority-context.md), + [ADR-022](022-handler-registration-provenance-and-composition-authority.md)) + — composition material, not directly callable from the wire. The MCP + server is reached over HTTP with a Bearer token from `Capabilities` + (the no-env-vars invariant), not by spawning a process that could read + the environment. +- `to_mcp` (expose local ops as MCP tools) serves an axum route with + Bearer auth middleware, matching the rmcp + `simple_auth_streamhttp.rs` pattern. An external MCP client (an + editor, an AI tool) discovers and calls alkhttp operations through + streamable HTTP, with alkhttp's auth/identity model applied at the + HTTP boundary. + +**Negative:** +- MCP servers that only support stdio (a significant fraction of the + current MCP ecosystem) cannot be consumed by `from_mcp` directly. The + operator runs a bridge (above). This is a deliberate exclusion, not a + feature gap. +- The "just download an MCP server and run it" UX that the MCP + ecosystem promotes is not supported. An alkhttp user who wants that UX + has to build the bridge and own the RCE risk. This is the correct + tradeoff for alkhttp's threat model, but it means alkhttp is not a + drop-in client for the stdio MCP ecosystem. + +## Assumptions + +1. **Streamable HTTP is the supported MCP transport in alkhttp.** This + is a one-way door: removing stdio support later (if it were ever + added) would break deployments that depend on it; not adding it is + the stable position. The streamable HTTP transport is the MCP + spec's network-isolated path and is what the rmcp examples use for + auth-gated servers. + +2. **The MCP ecosystem's stdio UX is not a target.** alkhttp is not + trying to be a drop-in client for "download untrusted MCP servers." + If a user wants that, the bridge approach puts the RCE risk + explicitly in their hands. + +3. **rmcp's streamable HTTP features are the right subset.** The + `mcp` feature gate pulls in `transport-streamable-http-client-reqwest` + (for `from_mcp`) and `transport-streamable-http-server` (for + `to_mcp`). The exact rmcp feature names are a two-way-door + implementation detail (rmcp may rename features across versions); the + one-way constraint is "streamable HTTP only, no stdio." + +## References + +- [ADR-014](014-secret-material-flow-and-capability-injection.md) — the + no-env-vars invariant; spawned processes reading env vars is the + pattern this ADR's exclusion prevents +- [ADR-015](015-privilege-model-and-authority-context.md) — + adapter-registered ops (`from_mcp`) are `Internal` by default +- [ADR-022](022-handler-registration-provenance-and-composition-authority.md) + — `from_mcp` provenance is a leaf +- alknet ADR-025 (vault local-only dispatch) — the analogous "dangerous + thing is not built by default; a separate crate with its own ADR" + pattern (textual reference; the vault decision record lives in the + alknet mono-repo's `docs/architecture/decisions/`) +- `docs/research/alknet-http/phase-0-findings.md` §4 (MCP stdio + exclusion) — alknet mono-repo research doc +- `/workspace/rust-sdk/` — MCP Rust SDK (rmcp v1.8.0); streamable HTTP + transport +- `/workspace/rust-sdk/examples/servers/src/simple_auth_streamhttp.rs` — + streamable HTTP MCP server with Bearer auth (the `to_mcp` pattern) +- `/workspace/rust-sdk/examples/clients/src/streamable_http.rs` — + streamable HTTP MCP client (the `from_mcp` pattern) +- `docs/architecture/http-mcp.md` — this crate's spec that implements + `from_mcp`/`to_mcp` + +## Port notes + +- Renames: "alknet-http" → alkhttp throughout (crate name, security + posture, feature-flag and threat-model references). +- alknet ADR-025 (vault local-only dispatch) is cited textually: it is + not ported to alkhttp or alkcall; the vault decision record remains in + the alknet mono-repo. ADR-014/015/022 are ported to this crate under + the same numbers and linked. +- The `client-and-adapters.md` citation became a textual "the alkcall + crate's `client-and-adapters.md`" reference (no relative link; the + document lives in alkcall's docs/architecture/). +- `crates/http/http-mcp.md` → `docs/architecture/http-mcp.md` (this + crate's docs/architecture/). The alknet mono-repo research-doc path + (`docs/research/alknet-http/phase-0-findings.md`) is kept and labeled + as a mono-repo doc. +- MCP server/client terminology untouched (inherent MCP directionality). +- No frontmatter in the source; status kept as Proposed. +- No decision content changed — the streamable-HTTP-only decision, the + stdio exclusion rationale, the bridge posture, the feature-gate shape, + and the consequences/assumptions are verbatim from the alknet ADR + modulo the renames logged above. \ No newline at end of file diff --git a/docs/architecture/decisions/039-http-server-and-client-host-colocated.md b/docs/architecture/decisions/039-http-server-and-client-host-colocated.md new file mode 100644 index 0000000..896d14b --- /dev/null +++ b/docs/architecture/decisions/039-http-server-and-client-host-colocated.md @@ -0,0 +1,180 @@ +# ADR-039: HTTP Server and Client Host Colocated in alkhttp + +*Ported from alknet ADR-039 (HTTP Server and Client Host Colocated in alknet-http); re-targeted to alkhttp.* + +## Status + +Proposed + +## Context + +alkhttp has two roles: an HTTP server (the `HttpAdapter` +`ProtocolHandler` for `h2`/`http/1.1`, built on `axum`/`hyper`) +and an HTTP client host (the `from_openapi`/`from_mcp` forwarding +handlers, built on `reqwest`). The question is whether these two +directions live in one crate (alkhttp) or are split into two +crates (an HTTP-server crate + an HTTP-client crate). + +ADR-003 lists the HTTP crate as a single crate with dependency +`alkcall, axum` and justifies the per-handler-crate decomposition +with "each handler is self-contained — it receives a byte stream and +manages its own protocol." That rationale covers the server side (the +`HttpAdapter` is self-contained), but it does not address the +within-crate dual-role question: should the inbound HTTP server and +the outbound HTTP client (the adapter forwarding handlers) be +colocated, or split? + +This is a load-bearing choice. Once published, downstream consumers +build import paths against the crate boundary; the shared `reqwest::Client` +and the no-env-vars invariant boundary (ADR-014) are scoped by it; the +`to_openapi`/`to_mcp` projections are pure-registry-consumers that +*describe* the server surface but live where the adapter types do. +Splitting later would be a rewrite of every consumer's import paths, +not a cheap revert. It needs an ADR. + +## Decision + +**One crate — alkhttp houses both the HTTP server and the HTTP +client host (the adapter forwarding handlers and the `to_*` projections).** + +The two directions share the HTTP dependencies and HTTP-specific +concerns that make splitting them counterproductive: + +- **Shared HTTP dependencies.** Both `axum` (server) and `reqwest` + (client) pull in `hyper`, `http`, `http-body`, `rustls`/TLS stack + types, and the HTTP header/status code types. A split into two crates + would either duplicate these dependencies across both crates or + force a third shared-types crate, neither of which is an improvement. +- **Shared HTTP-specific concerns.** Both directions care about HTTP + headers, status codes, content types, SSE framing, streaming vs + non-streaming bodies, and TLS trust stores. The `from_openapi` + forwarding handler's error mapping (HTTP status → `HTTP_` + error codes, ADR-023) and the `to_openapi` projection's error mapping + (`ErrorDefinition.http_status` → HTTP response status) are *the same + mapping* read in two directions — splitting them would put the two + halves in different crates. +- **The `to_*` projections describe the server surface.** `to_openapi` + generates an OpenAPI doc whose paths mirror the gateway HTTP routes + the `HttpAdapter` serves (ADR-036's mapping, superseded by ADR-047 — + see [ADR-047](047-remove-direct-call-http-surface.md)). + `to_mcp` exposes the + same operations as MCP tools. These projections consume the + `OperationRegistry` and produce specs; they live with the adapter + types (in alkhttp, per the adapter location map — see + the alkcall crate docs, client-and-adapters) + because they share the operation-spec→HTTP mapping logic with the + server's request dispatch. +- **The no-env-vars invariant boundary is crate-scoped.** The + `from_openapi`/`from_mcp` forwarding handlers are the credential + injection point (ADR-014). The invariant — "no handler reads outbound + credentials from any source other than `OperationContext.capabilities`" + — is verified against the handler implementations in this crate. A + split would put the invariant verification boundary across two crates. + +### What this does NOT change + +- ADR-003's rule "no handler crate depends on another handler crate" + applies to peer handler crates (alkhttp does not depend on + `alknet-ssh`). The alkhttp → `alkcall` edge is the + protocol-foundation exception (ADR-003 Amendment 1). This ADR is + about the *internal* structure of alkhttp, not its dependency + edges. +- The adapter location map (the `OperationAdapter` trait in + `alkcall`; the HTTP-backed adapter implementations in + alkhttp) is unchanged. This ADR records *why* the HTTP-backed + adapters live in the same crate as the HTTP server, not whether they + live in alkhttp vs `alkcall`. + +## Consequences + +**Positive:** +- One crate, one set of HTTP dependencies, one HTTP-specific concern + surface. No duplicated `hyper`/`http` types across two crates, no + shared-types crate needed. +- The `to_*` projections live with the server whose surface they + describe, and with the adapter types they consume. The operation-spec + → HTTP mapping logic is in one place. +- The no-env-vars invariant verification boundary is one crate. The + `from_openapi`/`from_mcp` handlers and the credential injection + logic they share are co-located. +- A downstream consumer wires one crate (alkhttp) into the + `HandlerRegistry` and gets the full HTTP surface — server + adapters + + projections. No two-crate wiring. + +**Negative:** +- A deployment that only needs the HTTP server (no `from_openapi`/`from_ + mcp` forwarding) still compiles the `reqwest` dependency. Mitigated: + the `mcp` feature is already gated (ADR-037); the `from_openapi` + forwarding is always available but the `reqwest` client is only + constructed if a `from_openapi`/`from_mcp` adapter is registered at + assembly time. The dependency is compiled, the client is lazy. +- A deployment that only needs the HTTP client (e.g., an agent crate + that only uses `from_openapi` forwarding, no inbound HTTP) still + compiles `axum`/`hyper`. This is the rarer case — the agent crate + (alknet-agent) consumes `alkcall` directly for tool dispatch + and uses `from_openapi` via alkhttp's adapter, but doesn't + serve inbound HTTP itself. In practice, the agent deployment wires + alkhttp for the adapters and the CLI wires it for the server; + the compile cost is paid once per workspace, not once per deployment. +- The crate is larger than a single-direction crate would be. This is + the cost of colocating shared concerns; the alternative (two crates + + a shared types crate) is more crates, not less code. + +## Assumptions + +1. **The shared-HTTP-dependencies argument holds.** `axum` and + `reqwest` both pull in `hyper` and the `http` crate's types; the + shared types (headers, status codes, method, URI) are the same. If + a future version of `axum` or `reqwest` diverges its HTTP types + (e.g., `axum` moves to a different HTTP implementation), this + argument weakens. As of `axum` 0.7+ and `reqwest` 0.12+, both are + built on `hyper` 1.x and share `http` types. + +2. **The `to_*` projections share enough mapping logic with the server + to justify colocation.** The operation-spec → HTTP path/method/ + error-status mapping is the same in both directions. If the + projections turn out to be pure registry-consumers with no + HTTP-mapping logic (just spec serialization), the colocation + argument is weaker — but the current design (ADR-036, ADR-023) + has them sharing the mapping. + +## References + +- [ADR-003](003-crate-decomposition.md) — crate decomposition (this + ADR addresses the within-alkhttp dual-role question, not the + dependency edge; Amendment 1 covers the `alkcall` edge) +- [ADR-014](014-secret-material-flow-and-capability-injection.md) — + the no-env-vars invariant whose verification boundary is crate-scoped +- [ADR-017](017-call-protocol-client-and-adapter-contract.md) — the + adapter contract; `to_*` are projections +- [ADR-023](023-operation-error-schemas.md) — the error mapping shared + between `from_openapi` (status → code) and `to_openapi` (code → + status) +- [ADR-036](036-http-to-call-operation-mapping.md) — the HTTP path = + operation path mapping shared between server dispatch and `to_openapi` +- [ADR-037](037-mcp-stdio-transport-exclusion.md) — the `mcp` feature + gate +- `overview.md` — the crate overview (the inline rationale + for this decision is replaced by a pointer to this ADR) + +## Port notes + +- Renames: alknet-http → alkhttp; alknet-call → alkcall; the + hypothetical split-crate names (`alknet-http-server` + + `alknet-http-client`) genericized ("an HTTP-server crate + an + HTTP-client crate"). +- The Context line "for `h2`/`http/1.1`/`h3`" was corrected to + "for `h2`/`http/1.1`" — h3 is out of scope in alkhttp (ADR-069); + alkhttp serves HTTP/1.1 + HTTP/2 on the standard ALPNs. +- The `to_openapi` route description "whose paths mirror the + `/{service}/{op}` HTTP routes the `HttpAdapter` serves (ADR-036)" + was corrected: the direct-call `/{service}/{op}` surface was removed + by ADR-047, so the doc now says the paths mirror the gateway routes, + with ADR-036's mapping noted as superseded by ADR-047 (linked as + `decisions/047-remove-direct-call-http-surface.md`, ported by another + agent under the same number). +- ADR-014/017/023/036/037 are being ported with the same numbers by + other agents and are linked as `decisions/NNN-.md` (alknet + slugs). The mono-repo link `../crates/call/client-and-adapters.md` + became a textual "alkcall crate docs" reference; + `crates/http/overview.md` → `overview.md` in `docs/architecture/`. \ No newline at end of file diff --git a/docs/architecture/decisions/041-mcp-tool-gateway-pattern.md b/docs/architecture/decisions/041-mcp-tool-gateway-pattern.md new file mode 100644 index 0000000..002dc0e --- /dev/null +++ b/docs/architecture/decisions/041-mcp-tool-gateway-pattern.md @@ -0,0 +1,263 @@ +# ADR-041: MCP Tool-Gateway Pattern for to_mcp + +*Ported from alknet ADR-041 (MCP Tool-Gateway Pattern for to_mcp); re-targeted to alkhttp.* + +## Status + +Proposed + +## Context + +The current `to_mcp` spec (`docs/architecture/http-mcp.md`) describes +`to_mcp` as "exposes the local registry's `External` operations as MCP +tools" — one MCP tool per alkhttp operation. An LLM connecting to an +alkhttp node with 200 registered operations gets 200 MCP tools dumped +into its context. This is the **tool-bloat problem**: the LLM's context +is bloated with tools that are irrelevant to the current task, degrading +its reasoning and wasting context budget. + +### The problem in concrete terms + +The MCP `tools/list` response returns every tool the server exposes. +An MCP client (an editor, an AI tool) loads all of them into the LLM's +context as tool definitions. An alkhttp node exposing 200 operations +produces a `tools/list` response with 200 `Tool` structs, each with a +name, description, and `inputSchema` (JSON Schema). The LLM sees 200 +tool definitions whether it needs them or not. This is the same anti- +pattern as loading every man page into a shell's environment — +absurd, but it's what the naive one-tool-per-operation mapping produces. + +### The pattern that works + +The project already has two examples of a better pattern: + +1. **The `memory` tool** (opencode): read-only access to the underlying + session database. The LLM doesn't load all past sessions into + context — it calls `memory` with a search query when it needs to + recall something. One tool, access to a large dataset on demand. +2. **The `worktree` tool** (opencode): gates 8-10 sub-tools behind a + single `worktree` entry point. The LLM has one tool in context; the + sub-tools are discovered and invoked through it. + +The general principle (same as Linux's `man` command): **don't load all +documentation/tools into context 24/7; expose a small fixed set of +meta-tools that gate access to the full set on demand.** + +### The call protocol's discovery surface + +The call protocol already has the discovery primitives that make this +work: + +- `services/list` — lists registered operations (filtered by + `AccessControl`). +- `services/schema` — returns an operation's `OperationSpec` + (input/output JSON Schemas, error schemas). + +The `to_mcp` gateway exposes these primitives (plus invocation) as a +small fixed set of MCP tools. The LLM searches for what it needs, learns +the schema, then calls — instead of having every operation pre-loaded. + +## Decision + +### 1. `to_mcp` exposes a fixed gateway tool set, not one tool per operation + +`to_mcp` exposes a small fixed set of MCP tools that gate access to the +full operation registry. The LLM has a few tools in context (not +hundreds); it discovers and invokes operations through the gateway. + +The gateway tool set (initial, two-way-door extensible): + +| MCP tool | Call protocol operation | Purpose | +|----------|------------------------|---------| +| `search` | `services/list` | List/search available operations (filtered by the caller's `AccessControl`). The LLM discovers what it can call. | +| `schema` | `services/schema` | Get an operation's `OperationSpec` (input/output JSON Schemas, error schemas). The LLM learns how to call a specific operation. | +| `call` | `call.requested` (Query/Mutation) | Invoke an operation by name with a JSON input. Returns the operation's output (or a typed error per the alkcall crate's ADR-016, `decisions/016-operation-error-schemas.md`). | +| `batch` | multiple `call.requested` | Invoke multiple operations in one tool call (correlated request IDs, OQ-14). The LLM batches independent calls. | + +Four tools. The LLM calls `search` to find operations relevant to its +task, `schema` to learn the input shape, `call` to invoke. Same pattern +as `man ` — discover on demand, don't preload. + +Sub (streaming responses) AND Pub (streaming requests, alkcall ADR-046) +operations are both excluded from to_mcp — MCP tool calls are +request/response. + +### 2. Sub operations are excluded from the MCP gateway + +MCP tool calls are request/response — an LLM invokes a tool and +receives a result. The call protocol's `Sub` type +(streaming, many `call.responded` events) does not map onto the MCP +tool-call model. The gateway exposes only `Query` and `Mutation` +operations (request/response). `Sub` operations are filtered +out of `search` results and cannot be invoked via `call`. + +This is a deliberate scoping decision, not a deferral: MCP tool calls +are request/response by protocol design; streaming subscriptions are a +different interaction model that doesn't fit the LLM tool-call pattern. +If a future MCP extension adds streaming tool calls, the gateway could +expose `Sub` operations through it — but that's a future MCP +spec question, not an alkhttp decision. + +### 3. `search` returns names + descriptions, not full schemas + +The `search` tool (backed by `services/list`) returns operation names, +namespaces, types, and short descriptions — not the full input/output +JSON Schemas. This keeps the search result small (the LLM is choosing +what to call, not how to call it yet). The LLM calls `schema` for the +specific operation it wants to invoke, getting the full `OperationSpec` +only when needed. Two-step discovery: search (cheap, list) → schema +(targeted, full spec). + +### 4. `call` maps to the call protocol's request/response dispatch + +The `call` tool takes `{ operation: "/fs/readFile", input: { ... } }` +and dispatches through the `OperationRegistry::invoke()` — the same +dispatch path the HTTP server uses +([ADR-036](decisions/036-http-to-call-operation-mapping.md)). The result +is mapped to an MCP `CallToolResult` (`structuredContent` for the +output, or `isError: true` for a `CallError` with the typed `details` +payload per the alkcall crate's ADR-016, +`decisions/016-operation-error-schemas.md`). The `batch` tool takes an +array of `{ operation, input }` pairs and returns an array of results. + +### 5. `AccessControl` gates the gateway + +The `search` tool's results are filtered by the caller's +`AccessControl::check(identity)` — the LLM (authenticated by bearer +token, [ADR-034](034-outgoing-only-x509-and-three-peer-roles.md) §4) +sees only the operations it is authorized to call. +The `call` tool's dispatch runs the same `AccessControl` check. An +LLM that calls `call` with an operation it isn't authorized for gets +`FORBIDDEN` (mapped to an MCP error result). The gateway does not +bypass the call protocol's authorization — it's the same dispatch +path, just reached through an MCP tool call instead of an HTTP request. + +## Consequences + +**Positive:** +- The LLM has 4 tools in context, not hundreds. Context budget is + preserved for the actual task; the LLM discovers operations on + demand through `search` + `schema`. This is the same pattern that + makes the `memory` and `worktree` tools effective. +- The gateway maps onto the call protocol's existing discovery + primitives (`services/list`, `services/schema`) and dispatch + (`OperationRegistry::invoke`). No new call-protocol mechanisms + needed — `to_mcp` is a thin wrapper around the existing surface. +- `AccessControl` gates the gateway. An LLM sees only what it's + authorized to call; the gateway doesn't leak operation existence or + schemas to unauthorized callers. +- `Sub` exclusion is explicit. The LLM tool-call model is + request/response; streaming doesn't fit, and pretending it does + would produce a broken mapping. + +**Negative:** +- The LLM needs two round-trips to call an operation it hasn't seen + before (`search` → `schema` → `call`). A one-tool-per-operation + mapping would let it call directly. The tradeoff: 4 tools in context + + 2 discovery round-trips vs. 200 tools in context + 0 round-trips. + The context budget is the scarcer resource; the round-trips are + cheap (the MCP server is local or nearby). +- The `search` tool's result format (names + descriptions, not full + schemas) means the LLM may need to call `schema` for multiple + operations before finding the right one. Mitigated: `search` can + accept a query/filter (namespace, keyword) to narrow results. +- The gateway tool set is fixed (4 tools). An operation that wants a + custom MCP tool (e.g., a specialized `git_clone` tool with a curated + input schema, not the generic `call` wrapper) is not exposed through + the gateway. A future "custom tool" extension could allow operations + to declare an MCP tool projection — but the gateway pattern is the + default, and the custom-tool path is additive (not a replacement). + +## Assumptions + +1. **The LLM context budget is the scarcer resource.** The tradeoff + favoring 4 tools + discovery round-trips over 200 preloaded tools + assumes the LLM's context window is more valuable than the network + round-trips. This holds for current LLMs (context windows are + large but not unlimited; tool definitions consume context + proportionally to their schemas). + +2. **`Query` and `Mutation` cover the LLM tool-call use case.** + LLMs invoke tools in a request/response pattern: call a tool, + receive a result, reason about it. Streaming subscriptions + (`call.responded` events over time) don't fit this pattern — the + LLM expects one result per tool call. The assumption is that the + operations an LLM wants to call are `Query`/`Mutation`, not + `Sub`. + +3. **The gateway tool set is stable.** Once LLM clients build + prompts/workflows against the `search`/`schema`/`call`/`batch` + tool set, changing the tool surface (renaming, removing) breaks + them. Adding tools is additive (non-breaking); removing or renaming + is a one-way door. The initial 4-tool set is the published contract. + +4. **`AccessControl` filtering is sufficient for `search`.** The LLM + sees the operations it's authorized to call. If an operation's + existence is itself sensitive (the LLM shouldn't know it exists + even if it can't call it), `Visibility::Internal` + ([ADR-015](015-privilege-model-and-authority-context.md)) is the + mechanism — Internal ops are excluded from `services/list` and + therefore from `search` results. The gateway does not add a + separate visibility layer. + +## References + +- [ADR-015](015-privilege-model-and-authority-context.md) — + External/Internal visibility (Internal ops excluded from + `services/list`, therefore from `search`) +- [ADR-017](017-call-protocol-client-and-adapter-contract.md) — + `to_*` adapters are projections (consume the registry, don't + produce entries) +- the alkcall crate's ADR-016 (`decisions/016-operation-error-schemas.md`) + — typed error `details` mapped to MCP error results +- [ADR-034](034-outgoing-only-x509-and-three-peer-roles.md) §4 — + browsers/MCP clients are not call-protocol peers (bearer token, no + `PeerId`) +- [ADR-036](decisions/036-http-to-call-operation-mapping.md) — the + HTTP-to-call dispatch path the `call` tool reuses +- [ADR-037](037-mcp-stdio-transport-exclusion.md) — streamable HTTP + only (the transport `to_mcp` uses) +- `docs/architecture/http-mcp.md` — this crate's spec that implements + the gateway +- `/workspace/rust-sdk/crates/rmcp/src/model/tool.rs` — the MCP + `Tool` struct (name, description, input_schema, output_schema) +- `/workspace/rust-sdk/crates/rmcp/src/handler/server.rs` — + `list_tools` / `call_tool` server trait (the interface `to_mcp` + implements) + +## Port notes + +- Renames: "alknet" → "alkhttp" where it referred to the crate/node + ("an alknet operation" → "an alkhttp operation"; "an alknet node" → + "an alkhttp node"; "not an alknet decision" → "not an alkhttp + decision"). +- **`Subscription` → `Sub`** throughout (alkcall rename; the + `OperationType` enum value). +- **MCP gateway exclusion widened (extension note):** the alknet + original excluded only `Subscription` operations. In alkhttp, + alkcall ADR-046 added `OperationType::Pub` (streaming requests, + producer→consumer). One line is added under §1: Sub (streaming + responses) AND Pub (streaming requests, alkcall ADR-046) operations + are both excluded from to_mcp — MCP tool calls are request/response. + §2 keeps the original `Subscription`-exclusion decision text, renamed + to `Sub`; the four-tool gateway set (search/schema/call/batch) is + unchanged. +- **Producer/consumer terminology**: §5 "browsers/MCP clients are not + alknet peers" → "browsers/MCP clients are not call-protocol peers" + (the ADR-034 peer-roles framing). HTTP/MCP server-client + directionality untouched (MCP client/server, `tools/list`, + `transport-*` names are protocol-inherent). +- **Cross-refs**: `../../decisions/...` → `decisions/...`; ADR-036 is + ported to this crate under the same number and linked as + `decisions/036-http-to-call-operation-mapping.md` (alknet slug). + The source cited alknet ADR-023 (operation error schemas) — its + alkcall decision record is ADR-016, cited textually per the + alkcall-cited-decisions convention. +- `crates/http/http-mcp.md` → `docs/architecture/http-mcp.md` (this + crate's docs/architecture/). +- No frontmatter in the source; status kept as Proposed. +- No decision content changed — the 4-tool gateway set, the + request/response exclusion rule, the two-step discovery shape, the + dispatch/`AccessControl` mapping, and the consequences/assumptions + are verbatim from the alknet ADR modulo the renames and the logged + Pub-exclusion extension line. \ No newline at end of file diff --git a/docs/architecture/decisions/042-openapi-gateway-pattern.md b/docs/architecture/decisions/042-openapi-gateway-pattern.md new file mode 100644 index 0000000..6bb43ad --- /dev/null +++ b/docs/architecture/decisions/042-openapi-gateway-pattern.md @@ -0,0 +1,335 @@ +# ADR-042: OpenAPI Gateway Pattern for to_openapi + +*Ported from alknet ADR-042 (OpenAPI Gateway Pattern for to_openapi); re-targeted to alkhttp.* + +## Status + +Proposed + +## Context + +The current `to_openapi` spec (`http-adapters.md` in this crate's +`docs/architecture/`; alknet original: `crates/http/http-adapters.md`) +describes `to_openapi` as generating a traditional OpenAPI document with +one path +entry per `External` operation — `POST /fs/readFile`, +`POST /agent/chat`, etc., each with parameters, request body, and +responses built from the operation's `input_schema`/`output_schema`/ +`error_schemas`. This is the "inverse of `from_openapi`" framing: since +`from_openapi` merges OpenAPI path params / query params / request body +into a single flat JSON input schema, `to_openapi` should split them +back out. + +### The flat→structured problem + +The inverse is genuinely messy. The call protocol's input is a flat JSON +object (e.g., `{ path, content, encoding }` for `fs/writeFile`). To +generate a traditional OpenAPI path entry (`POST /fs/{path}` with path +param `path`, body `content`), `to_openapi` would need to know which +fields are path params, which are query params, and which is the body. +That information isn't in the flat schema — it's metadata the call +protocol doesn't carry because it doesn't care about HTTP parameter +structure. `to_openapi` would need either: + +1. HTTP-specific metadata on `OperationSpec` (which fields are path + params, etc.) — a leaky abstraction that puts HTTP concerns in the + protocol-foundation crate (alkcall), or +2. Heuristics (guess that fields named `id` are path params?) — fragile + and wrong, or +3. Manual annotation per operation — boilerplate that defeats the "pure + projection" promise. + +All three are messy. The flat→structured split is the hard direction, +and it's the one `to_openapi` has to do. + +### The per-caller API surface problem + +A traditional OpenAPI document is static — it describes the full API +surface regardless of who's reading it. Real APIs have per-caller +authorization: an admin sees admin operations, a regular user sees a +subset. OpenAPI has no standard mechanism for "show me only what I have +access to." The Gitea API is a concrete failure case: its OpenAPI spec +dumps the full API (including admin operations) to every caller, +regardless of privilege. A user reading the spec can't tell which +endpoints they can actually call without trial-and-error `403`s. + +The call protocol already has the per-caller filtering primitive: +`services/list` is `AccessControl::check(identity)`-filtered — the +caller sees only the operations they are authorized to call. A +`to_openapi` that generates a static full-surface doc loses this +property. A `to_openapi` that uses the gateway pattern preserves it. + +### The pattern that works + +The same tool-gateway pattern ADR-041 applies to `to_mcp` applies here: +`to_openapi` exposes a small fixed set of endpoints that gate access to +the full operation registry. The external client (a code generator, a +human developer, a `fetch`-based client) calls `search` to discover +operations, `schema` to learn an operation's input shape, `call` to +invoke. The input is always a flat JSON body — no path/query/body split +to reverse-engineer. JSON Schema for the input/output is already in the +`OperationSpec` — no conversion beyond wrapping it in OpenAPI's schema +format. + +The OpenAPI gateway has one endpoint the MCP gateway doesn't: +`subscribe` (SSE). OpenAPI/SSE supports streaming; MCP tool calls don't. +So the OpenAPI gateway is 5 endpoints; the MCP gateway is 4. (In +alkhttp, the OpenAPI gateway is extended with `/publish` — ADR-068 — +making it 6 endpoints; see Decision §1.) + +## Decision + +### 1. `to_openapi` exposes a fixed gateway endpoint set, not one path per operation + +`to_openapi` generates an OpenAPI document with a small fixed set of +endpoints that gate access to the full operation registry. The external +client discovers and invokes operations through the gateway. + +The gateway endpoint set (initial, two-way-door extensible): + +| OpenAPI path | Call protocol operation | HTTP method | Purpose | +|--------------|------------------------|-------------|---------| +| `/search` | `services/list` | `GET` | List/search available operations (filtered by the caller's `AccessControl`). Returns names + descriptions. | +| `/schema` | `services/schema` | `GET` | Get an operation's full `OperationSpec` (input/output JSON Schemas, error schemas). | +| `/call` | `call.requested` (Query/Mutation) | `POST` | Invoke an operation by name with a JSON input. Returns the output or a typed error ([ADR-023](023-operation-error-schemas.md)). | +| `/batch` | multiple `call.requested` | `POST` | Invoke multiple operations in one request (correlated request IDs, alknet OQ-14). Returns an array of results. | +| `/subscribe` | `call.requested` (Sub) | `POST` (SSE) | Invoke a streaming operation. Body `{ operation, input }` (same shape as `/call`); response is `text/event-stream` — each `call.responded` is an SSE frame, `call.completed` closes the stream. | + +Five endpoints (the 5-endpoint set, extended with `/publish` in alkhttp +— ADR-068; `POST /publish` invokes Pub operations — producer→consumer +streaming; the request body is streamed as newline-delimited JSON, each +line one published chunk, and the final `ResponseEnvelope` is returned +as the HTTP response). The client calls `/search` to find operations, +`/schema` to learn the input shape, `/call` (or `/subscribe` for +streaming, `/publish` for Pub streaming) to invoke. The input is always +a flat JSON body (`{ operation: "/fs/readFile", input: { ... } }`); the +output is the operation's result as JSON. No path/query/body split to +reverse-engineer. + +### 2. `subscribe` is the OpenAPI gateway's streaming endpoint (SSE) + +The OpenAPI gateway includes `subscribe` (which the MCP gateway excludes +— ADR-041, MCP tool calls are request/response). The `subscribe` +endpoint maps `Sub` operations onto SSE: `POST /subscribe` with +a `{ operation, input }` JSON body (same shape as `/call`) and +`Accept: text/event-stream`, each `call.responded` event is an SSE +`data:` frame, `call.completed` closes the stream, `call.aborted` closes +with an error frame. This is the same SSE projection ADR-036 describes +for `h2`/`http/1.1` clients — the gateway's `subscribe` endpoint is the +single SSE entry point instead of per-operation SSE streams. + +`POST` (not `GET`) is used because `/subscribe` is an invoke endpoint +that carries `{ operation, input }` in the request body, the same flat +JSON body shape the rest of the gateway uses. A `GET` request has no +body, so it cannot carry the operation name and input. The SSE response +is negotiated via `Accept: text/event-stream` on the `POST`, not via the +method. (Browsers using `EventSource` cannot `POST`, but browsers use +WebSocket for the bidirectional path — ADR-044; the HTTP gateway's +`/subscribe` is for non-browser HTTP clients, and `fetch` + +`ReadableStream` handles POST-SSE cleanly.) + +### 3. The generated OpenAPI doc is per-caller (AccessControl-filtered) + +The `/search` endpoint's results are filtered by the caller's +`AccessControl::check(identity)` — the client sees only the operations +it is authorized to call. The `/call` and `/subscribe` endpoints run the +same `AccessControl` check on dispatch. The generated OpenAPI doc +describes the gateway endpoints (5 fixed paths — extended with +`/publish` in alkhttp, ADR-068); the per-caller +operation surface is discovered through `/search`, not preloaded into +the doc. + +This is the key advantage over a traditional per-operation-paths OpenAPI +doc: the per-caller API surface is the default, not an afterthought. A +client reading the gateway OpenAPI doc learns the gateway's shape (5 +endpoints, stable — 6 in alkhttp); a client calling `/search` learns +what *it* can call (per-caller, AccessControl-filtered). The Gitea +failure mode (dumping admin ops to every caller) is structurally +impossible — `/search` doesn't return operations the caller can't call. + +### 4. The gateway OpenAPI doc is a compatibility contract + +Once published, the gateway endpoint set (5 endpoints — extended with +`/publish` in alkhttp, ADR-068) and the +request/response shapes are a compatibility contract ([ADR-017](017-call-protocol-client-and-adapter-contract.md) +Consequences). Adding endpoints is additive (non-breaking); removing or +renaming is a one-way door. The initial 5-endpoint set is the published +contract. The versioning strategy for the generated doc was tracked as +alknet OQ-39 (now **resolved by +[ADR-045](045-to-openapi-gateway-spec-versioning.md)**: +`info.version` semver tracks the gateway endpoint contract, not the +operation set) — the gateway pattern simplifies versioning to 5 stable +endpoints instead of a per-operation surface. + +### 5. A traditional per-operation-paths projection is additive, not replacement + +A deployment that wants a traditional REST OpenAPI doc (per-operation +paths with split parameters) can build it as a separate projection with +the HTTP-specific metadata (which fields are path params, etc.). The +gateway pattern is the default `to_openapi` projection; the traditional +projection is an additive alternative for deployments that need it. The +gateway does not foreclose the traditional projection — it just doesn't +require it for the common case. + +## Consequences + +**Positive:** +- No flat→structured split. The gateway's input is always a flat JSON + body (`{ operation, input }`); the operation's input/output schemas + are already JSON Schemas in the `OperationSpec`. No reverse- + engineering of path/query/body semantics. The messy direction of the + `from_openapi` inverse is sidestepped. +- Per-caller API surface by default. `/search` is + `AccessControl`-filtered; the client sees only what it can call. The + Gitea failure mode (dumping admin ops to every caller) is structurally + impossible. This is a property the traditional per-operation-paths + OpenAPI doc cannot provide (OpenAPI has no per-caller filtering + concept). +- Easy to build clients for. Any language's `fetch` + JSON Schema + libraries can call the gateway: `POST /call` with a JSON body, get a + JSON result. No code generator needed for the common case; a code + generator produces a `CallClient` (call/search/schema/batch/ + subscribe) instead of typed per-operation methods. +- 5 stable endpoints instead of a per-operation surface (6 in alkhttp + with `/publish`, ADR-068). The + versioning concern (alknet OQ-39) is simpler — a small set of + endpoints that rarely change vs. a per-operation surface that changes + on every operation addition/modification. +- `subscribe` maps cleanly onto SSE — the same projection ADR-036 + describes, just as a single gateway entry point instead of per- + operation SSE streams. +- A deployment that wants the traditional REST surface can build it + additively. The gateway doesn't foreclose it. + +**Negative:** +- The generated OpenAPI doc is not a "nice UI" by default. A Swagger UI + rendering shows 5 generic endpoints instead of a REST tree. This is + the tradeoff for avoiding the flat→structured split and gaining per- + caller filtering. A deployment that wants the nice UI builds the + traditional projection (additive, with metadata). +- A code generator reading the gateway OpenAPI doc produces a + `CallClient` (generic call/search/schema methods), not typed per- + operation methods. Typed methods require the traditional projection + (with metadata) or a client that reads `/search` + `/schema` and + generates typed wrappers at build time. The gateway is optimized for + the `fetch`-and-JSON-Schema use case, not the code-generation use + case. +- The gateway doc is less "traditional" — a developer expecting a + REST OpenAPI doc sees a small RPC-style surface instead. This is + honest (the call protocol is a flat JSON RPC, not a REST API), but + it's a departure from OpenAPI conventions. + +## Assumptions + +1. **The gateway endpoint set is stable.** Once external clients build + against the 5-endpoint gateway, changing the endpoint set (renaming, + removing) breaks them. Adding endpoints is additive (non-breaking). + The initial 5-endpoint set is the published contract (alkhttp + extends it with `/publish` — ADR-068 — an additive addition, the + pattern this assumption already permits). + +2. **`AccessControl` filtering is the right per-caller mechanism.** The + client sees the operations it's authorized to call. If an operation's + existence is itself sensitive, `Visibility::Internal` + ([ADR-015](015-privilege-model-and-authority-context.md)) is + the mechanism — Internal ops are excluded from `services/list` and + therefore from `/search` results. The gateway does not add a + separate visibility layer. + +3. **The common case is `fetch` + JSON Schema, not code generation.** + The gateway is optimized for the developer who calls `POST /call` + with a JSON body and parses the result. The code-generation case + (typed per-operation methods) is served by the traditional projection + (additive) or a client that generates wrappers from `/search` + + `/schema` at build time. + +4. **`subscribe` (SSE) is the streaming projection for the gateway.** + Over `h2`/`http/1.1`, subscriptions are SSE. Over WebSocket (the v1 + browser bidirectional path, ADR-044), subscriptions project onto the + WS connection directly as binary messages — the gateway's `/subscribe` + is the `h2`/`http/1.1` SSE path; the WebSocket path is the native + call-protocol session (the alkcall crate's `docs/architecture/` + stream model; the gateway shape does not + appear on WS per [ADR-048](048-websocket-native-session-not-gateway.md)). + WebTransport (`h3`, deferred per ADR-044) would project onto + WebTransport streams; no deferred WebTransport spec is carried in + alkhttp (per ADR-044). + +## References + +- [ADR-015](015-privilege-model-and-authority-context.md) — + External/Internal visibility (Internal ops excluded from + `services/list`, therefore from `/search`; the decision record is + alkcall ADR-017) +- [ADR-017](017-call-protocol-client-and-adapter-contract.md) — + `to_*` adapters are projections; published-spec compatibility contract + (the decision record is alkcall ADR-022) +- [ADR-023](023-operation-error-schemas.md) — typed error `details` + mapped to OpenAPI error responses (the decision record is alkcall + ADR-016) +- [ADR-036](036-http-to-call-operation-mapping.md) — the SSE projection + for subscriptions over `h2`/`http/1.1` (the gateway's `/subscribe` + endpoint uses the same SSE framing) +- [ADR-044](044-defer-webtransport-browsers-use-websocket.md) — + WebSocket is the v1 browser bidirectional path; `h3`/WebTransport + deferred (the gateway's `/subscribe` is the `h2`/`http/1.1` SSE path; + the WS path is the native call-protocol session). ADR-038 is + superseded by ADR-044. +- [ADR-068](068-gateway-publish-endpoint.md) — the alkhttp extension of + this ADR's gateway: `POST /publish` (Pub operations, + newline-delimited JSON chunks) makes the OpenAPI gateway 6 endpoints +- ADR-041 (MCP Tool-Gateway Pattern for `to_mcp`) — the sibling gateway + pattern for `to_mcp` (4 tools; `subscribe` excluded because MCP tool + calls are request/response; ported to alkhttp — see + `decisions/041-mcp-tool-gateway-pattern.md` if present, otherwise the + alknet record) +- alknet OQ-39 — `to_openapi` published-spec versioning (simplified by + the gateway pattern to 5 stable endpoints; **resolved by + [ADR-045](045-to-openapi-gateway-spec-versioning.md)**) +- `http-adapters.md` (in this crate's `docs/architecture/`) — the spec + that implements the gateway (alknet original: + `crates/http/http-adapters.md`) + +## Port notes + +- Renames: "alknet" → alkhttp where the system/node being described is + meant; "alknet-call" (protocol-foundation crate) → alkcall. +- `OperationType::Subscription` → `Sub` in the gateway endpoint table, + §2, and throughout (alkcall rename). alkcall ADR-046 added + `OperationType::Pub` (producer→consumer streaming, `HandlerKind::Sink`); + the original 5-endpoint table predates `Pub` and is preserved verbatim + as decision history — the `/publish` extension is marked as an alkhttp + addition (ADR-068), never silently rewritten. +- **Gateway endpoint count**: the alknet record's "5 endpoints" is + preserved everywhere as the original decision; alkhttp's `/publish` + extension (ADR-068) is noted inline at each site where the count + matters (§1, §3, §4, Assumption 1, Context). "Never rewrite history" + framing: the published initial contract remains the 5-endpoint set; + `/publish` is an additive extension under the contract's own + additivity rule (§4, Assumption 1). +- Producer/consumer terminology: "an admin sees admin operations, a + regular user sees a subset" and similar retain their names — no + call-protocol server/client role framing existed in the original + body. The external client roles (code generator, human developer, + `fetch`-based client, Gitea-style API consumers) are inherent-directionality + roles of the HTTP/OpenAPI surface and keep their names. +- Cross-reference remappings: ADR-015/017/023 are ported to this crate + under the same numbers and linked; their alkcall record numbers are + noted alongside (015→alkcall ADR-017, 017→alkcall ADR-022, + 023→alkcall ADR-016). ADR-041 is cited textually with its alknet + number (the MCP gateway pattern; not part of this port's 3 files — + verify the alkhttp port's existence when linking). +- OQ references (OQ-14, OQ-39) are alknet OQ-tracker items, cited + textually; OQ-39 is resolved by [ADR-045](045-to-openapi-gateway-spec-versioning.md) + (ported to this crate under the same number). +- Spec-document links converted per alkhttp conventions: + `crates/http/http-adapters.md` → `http-adapters.md` in this crate's + `docs/architecture/`; `websocket.md`/`webtransport.md` were alknet + mono-repo spec-tree documents — the WS path here is the native + call-protocol session (ADR-048) and no deferred WebTransport spec is + carried (ADR-044). +- No decision content changed — the flat→structured problem, per-caller + surface problem, gateway pattern, the 5-endpoint table, the `subscribe` + SSE semantics, `POST`-not-`GET` rationale, compatibility-contract + clause, additive traditional projection, consequences, and assumptions + are verbatim from the alknet ADR modulo the adaptations above. \ No newline at end of file diff --git a/docs/architecture/decisions/044-defer-webtransport-browsers-use-websocket.md b/docs/architecture/decisions/044-defer-webtransport-browsers-use-websocket.md new file mode 100644 index 0000000..383574d --- /dev/null +++ b/docs/architecture/decisions/044-defer-webtransport-browsers-use-websocket.md @@ -0,0 +1,555 @@ +# ADR-044: Defer h3/WebTransport; Browsers Use WebSocket + +*Ported from alknet ADR-044 (Defer h3/WebTransport; Browsers Use WebSocket); re-targeted to alkhttp.* + +## Status + +Accepted (supersedes alknet ADR-038; parks alknet ADR-040, alknet ADR-043) + +## Status amendment (alkhttp port) + +- **§1 (WebSocket as the browser bidirectional path) stands** and is the + operative decision for this crate: the browser bidirectional path in + alkhttp is WebSocket (alkhttp ADR-048 defines what the WS session + carries). +- **The deferral mechanics are superseded by alkhttp ADR-069**: + WebTransport is not "deferred" in alkhttp — it is **removed from + alkhttp scope entirely** (an alknet concern). There is no `h3` + feature gate, no revival trigger, and no `webtransport.md` spec in + this crate. ADR-044's "defer, revive when the ALPN-stream-proxy use + case arrives" framing is an alknet posture and does not carry into + alkhttp. +- **alknet ADR-038, ADR-040, and ADR-043 are not ported to alkhttp.** + They remain alknet decision records (ADR-038 superseded by alknet + ADR-044; ADR-040/ADR-043 parked per alknet ADR-044). Nothing in this + crate implements or plans them; if WebTransport revives as an alknet + transport, the browser path through alkhttp remains WebSocket. + +## Context + +alknet ADR-038 brought `h3`/WebTransport into scope as a first-class HTTP +transport, framed against the "two-way door as deferral" anti-pattern +(alkcall ADR-032 §"What this framework is NOT"). alknet ADR-040 (the +ALPN-stream-proxy) and alknet ADR-043 (the bidirectional-substrate +reframing) extended it. Three ADRs, one crate-spanning spec +(`webtransport.md`), and a body of design work. + +Working through the implementation path surfaced a different concern than the +one alknet ADR-038 was written to correct. alknet ADR-038 correctly rejected *deferral- +as-hedging*; the present decision is *deferral-as-scoping*, which +alkcall ADR-032 explicitly permits (a decision that "genuinely doesn't +need to be made yet because the use case isn't concrete" — scope +management, not door-type classification). The two must not be +confused. Three concrete findings drove the scope re-evaluation: + +*(Findings 2 — the WebTransport standards/dependency-maturity analysis — +and the iroh-relay precedent below concern the alknet transport layer +where QUIC/TLS/h3 live. alkhttp carries no transport stack; it is +retained here for provenance of why the browser path is WebSocket.)* + +### Finding 1 — the browser bidirectional path doesn't require WebTransport + +The load-bearing use case for `h3`/WebTransport in v1 is **a browser reaching +the call protocol bidirectionally**. alknet ADR-043 §2 establishes that the call +protocol's bidirectionality applies unchanged over any bidirectional stream — +the `Dispatcher` is stream-agnostic (alkcall ADR-015). That property is not unique to +WebTransport streams. **WebSocket is a full-duplex, long-lived connection over +which either side can send framed messages**, and the call protocol's +`EventEnvelope` framing fits the WebSocket path cleanly (the +`call.requested`/`call.responded`/`call.completed`/`call.aborted` +exchange works over WebSocket with no protocol change — the same `Dispatcher`, +the same `PendingRequestMap`, the same correlation by request ID; see the +framing note below and alkhttp ADR-048's status amendment for the +current chunk/framing model). + +What WebTransport gives *over* WebSocket — native multiplexed bidirectional +streams, datagrams, the "carry any ALPN as a stream" substrate framing +(alknet ADR-043) — is genuinely better engineering, but none of it is *required* for +the call protocol from a browser. The call protocol multiplexes multiple calls +over a single connection by request ID (alkcall ADR-015); it does not need +WebTransport's per-stream multiplexing. The substrate/proxy framing (alknet +ADR-040, alknet ADR-043) is the thing that *does* benefit from WebTransport's +stream model — and that use case is the speculative one (see Finding 3). + +**Framing correction (alkhttp port):** the original Finding 1 stated +"one `EventEnvelope` = one WS binary message." In the current wire +model that is superseded: the WS message boundary carries **8-byte +chunks of the channels protocol**, and on channel 0 the call protocol's +`EventEnvelope` frames are carried as length-prefixed JSON (alkcall's +frame format, alkcall ADR-014) inside the chunk payload — not +one-envelope-per-WS-message. The load-bearing property (self-delimited +frames, correlation by request ID, no protocol change) is unchanged. +See alkhttp ADR-067 §framing and alkhttp ADR-048's status amendment. + +### Finding 2 — WebTransport is a draft standard on an experimental dependency stack *(alknet concern)* + +WebTransport over HTTP/3 is still an IETF draft (`draft-ietf-webtrans-http3`, +at `-07` at time of writing), not an RFC. The Rust implementation landscape is +correspondingly immature: + +- `wtransport` (the reference read during research) is a complete + pure-Rust implementation, but its own README states it "is not considered + completely production-ready" and "may undergo changes as the WebTransport + specification evolves." +- The hyperium stack (`h3` + `h3-quinn` + `h3-webtransport` + `h3-datagram`) + fits the axum/hyper ecosystem more naturally (h3 produces `http::Request` + types that axum consumes directly, which is load-bearing for the spec's + "HTTP/3 requests go through the same axum `Router`" commitment), but h3's + own README says it is "still very experimental... API could change." +- A research spike would be needed to verify the hyperium stack's + server-side WebTransport API before committing to it — the axum-bridge + feasibility is the load-bearing claim and is not yet confirmed against + actual crate APIs, only against READMEs and design philosophy. + +Either choice puts a draft-standard protocol and an experimental Rust +dependency on the security surface of alkhttp's first release. The `h3` +feature gate (alknet ADR-038) isolates the risk for non-browser-facing deployments, +but a browser-facing hub must enable it — so the risk is borne precisely by +the deployment shape that motivates having a browser path at all. + +*(In alkhttp this risk analysis is historical: the crate never gains an +`h3` feature. WebTransport is removed from scope per alkhttp ADR-069.)* + +### Finding 3 — the ALPN-stream-proxy is speculative; the call protocol is not + +alknet ADR-040 (the ALPN-stream-proxy — a browser with a WASM parser for SSH/SFTP/git +reaching any ALPN handler via WebTransport) is the genuinely compelling +WebTransport use case. It is also the one that is *not* required for v1: + +- The call protocol from a browser works over WebSocket (Finding 1). +- The downstream crates unlocked by completing alkhttp (the SSH, git, + SFTP crates) do not require WebTransport or the proxy. They expose their + ALPNs natively over QUIC; the proxy is a *browser reachability* feature + for those ALPNs, not a prerequisite for the ALPNs to exist. *(The + transports exposing those ALPNs are alknet concerns.)* +- The WASM parsers (the browser-side SSH/SFTP/git clients) are themselves + downstream artifacts not yet built. The proxy is only useful once a parser + exists to consume it. + +The proxy is "useful, and cheap-on-top *if* WebTransport already exists" — +but WebTransport does not yet exist, and building it speculatively to enable +a proxy whose consumers do not yet exist is the scope inversion. *(And in +the alkhttp crate, it was never built at all — removed from scope per +alkhttp ADR-069.)* + +### The iroh precedent *(alknet-adjacent, but the signal carries)* + +iroh's own relay (`iroh-relay`, the DERP-equivalent that provides NAT traversal +fallback) chose **WebSocket (WSS)**, not WebTransport, for its fallback path. +This is a strong signal from a project whose entire design center is QUIC and +P2P connectivity: when the question was "what does a browser need to reach our +protocol bidirectionally," their answer was WSS, not WebTransport. Aligning +with that precedent is not cutting against competent practice — it is +matching it. + +### Concrete prior art: `@alkdev/pubsub` + +The WebSocket path is not speculative — there is working prior art in the +same workspace. The `@alkdev/pubsub` package (`/workspace/@alkdev/pubsub/`) +already has a WebSocket client (`event-target-websocket-client.ts`) and +server (`event-target-websocket-server.ts`) built on a generalized "event +target" abstraction with an `EventEnvelope { type, id, payload }` shape. +The call protocol's `EventEnvelope` was derived from this envelope +(refined with typed event names `call.requested`/`call.responded`/etc. and +structured payloads); the sibling `@alkdev/operations` package +(`/workspace/@alkdev/operations/`) shares the lineage and uses the +`path.do.op` (dot-separated) vs the call protocol's `path/to/op` +(slash-separated) convention — a minor, mechanical delta. Syncing the +pubsub/operations WebSocket client to the call protocol's envelope is a +small adjustment (~a day of work: the envelope shape, the event-name +typing, the path separator), not a from-scratch browser-client build. +This is why the WebSocket path opens doors quickly: the browser (and +Node) client is mostly already written. *(In the current wire model the +pubsub-style clients additionally speak the channels chunk framing on +the WS path — see the framing correction under Finding 1 and alkhttp +ADR-067.)* + +### The tradeoff between two use cases, not "good enough for now" + +It is worth being precise about *why* WSS is the right choice here, because +"good enough until it isn't" undersells the decision. The two browser-reach +use cases have different right tools: + +- **The call protocol from a browser (bidirectional).** WSS is *genuinely + the right tool*, not a stopgap. The call protocol multiplexes by request + ID (alkcall ADR-015), not by stream — it does not need WebTransport's + per-stream multiplexing. A WebSocket is a full-duplex, long-lived, + framed-message channel; the call protocol's framing fits the WS path + cleanly (see the framing correction for the current chunk-based model). + For this use case, WebTransport's stream model is engineering + sophistication the call protocol has no use for. WSS is not "good + enough" — it is well-matched. +- **The generalized ALPN router/proxy (a browser reaching a non-call ALPN + — SSH/SFTP/git via WASM).** WebTransport's native multi-stream model is + *genuinely the right tool* here, and WSS is *probably worse* for it. A + browser reaching a non-call ALPN over WSS would have to multiplex + logical streams over one WS frame stream by application-level framing — + doable (alknet ADR-043 §"SSH/SFTP/git-over-WSS-from-a-browser is + technically possible"), but it re-implements at the application layer + what WebTransport gives at the transport layer. This is the use case + WebTransport was built for, and it is the speculative one (Finding 3) — + the consumers (WASM SSH/SFTP/git parsers) do not exist yet. *(In + alkhttp this second use case does not exist at all: WebTransport is + removed from scope, alkhttp ADR-069, and the ALPN-stream-proxy was + never in this crate.)* + +So the original deferral was not "use the worse tool now, upgrade to the +better tool later." It was "use the right tool for the use case we *have* +(call protocol from a browser → WSS), and defer building the tool for the +use case we *don't have yet* (generalized ALPN proxy → WebTransport)." +In the alkhttp port, the second half of that sentence is closed outright: +there is no deferred WebTransport future in this crate — the crate's +browser bidirectional path is WebSocket, full stop (alkhttp ADR-069). + +## Decision + +### 1. Defer `h3`/WebTransport. Browsers reach the call protocol over WebSocket. *(§1 stands; deferral mechanics superseded by alkhttp ADR-069)* + +The `h3` ALPN, the `h3` feature gate, and the WebTransport dependency stack +are **deferred** in the original alknet decision — not implemented in the +initial release. In **alkhttp** this resolves further: WebTransport is +**removed from the crate's scope entirely** (alkhttp ADR-069); there is no +`h3` ALPN, no `h3` feature, and no deferred revival in this crate. The +clause that stands unchanged is the browser path itself: a browser +connecting to a hub authenticates by bearer token and upgrades an +HTTP/1.1 or HTTP/2 request to WebSocket. The resulting full-duplex WS +connection carries the call protocol's `EventEnvelope` frames (in the +current wire model: length-prefixed JSON frames on channel 0 inside the +channels 8-byte chunk framing — see the framing correction under Finding +1 and alkhttp ADR-067/ADR-048). The browser is a bidirectional +call-protocol client over this connection, using the same `Dispatcher` +and `PendingRequestMap` as the `alk/call` QUIC path (alkcall ADR-015 — +stream-agnostic correlation; a WS message stream is just another +`BiStream`-satisfying transport, extending the stream-agnostic claim +from QUIC bidirectional streams to any framed full-duplex byte channel). + +The original scope-decision framing (deferral-as-scoping, per alkcall +ADR-032) still describes why WS was chosen; the reversal trigger below +does not apply to alkhttp — the "revival" it describes would be an +alknet concern and would not reopen alkhttp's surface. + +### 2. alknet ADR-038 is superseded by this ADR. *(alknet record; not ported)* + +alknet ADR-038's core decision — that `h3` is in scope, not deferred — is +reversed by this ADR in alknet. alknet ADR-038's *correction* of the +"two-way-door-as-deferral" anti-pattern stands as a document (the +anti-pattern is real); its specific decision (h3 in scope now) is +superseded. alknet ADR-038 is marked Superseded in the alknet record. It +is **not ported to alkhttp**; this crate has no `h3` decision to record +browsers-transport work against. + +### 3. alknet ADR-040 and ADR-043 are parked, not superseded. *(alknet records; not ported)* + +alknet ADR-040 (the ALPN-stream-proxy) and alknet ADR-043 (the +bidirectional-substrate reframing) are **not superseded** in the alknet +record — their decisions are correct, and they revive unchanged when +WebTransport revives *as an alknet transport*. They are marked Proposed +with an amendment noting implementation is deferred per this ADR. In +alkhttp, neither is ported and neither has any footprint. The two +transfers that the original decision applied during deferment: + +- **alknet ADR-043 §2 (call-protocol bidirectionality over WebTransport) + transfers to WebSocket unchanged.** WebSocket is full-duplex; the call + protocol's bidirectionality applies over a WS connection exactly as + alknet ADR-043 §2 describes for WebTransport. The browser case where + the client registers no ops remains a use-case scoping, not an + architectural limitation. *(This transfer is the part of ADR-044 that + is live in alkhttp — it is what alkhttp ADR-048 implements.)* +- **alknet ADR-043 §3 (the no-`PeerId` connection-local overlay) + transfers to WebSocket unchanged.** A browser over WSS has no `PeerId` + on the hub's side for the same reasons it has none over WebTransport + (see §5 below); the connection-local Layer 2 overlay applies + (alkcall ADR-019). The pattern is transport-agnostic. + +What does *not* transfer to WebSocket is alknet ADR-040 (the +ALPN-stream-proxy) and alknet ADR-043 §4 (the non-call-ALPN substrate +mechanism). Those require WebTransport's stream model and revive with +it *(as alknet work; alkhttp has no such path)*. SSH/SFTP/git-over-WSS- +from-a-browser is technically possible (multiplex logical streams over +one WS frame stream) but is not specified here — it is the same +speculative use case that motivated deferring WebTransport, and it is +not needed for v1. + +### 4. WebSocket is the browser bidirectional path; HTTP/1.1+HTTP/2 remain the one-directional projection. + +alkhttp's browser-reachable surface is: + +| Transport | Direction | Use case | +|-----------|-----------|----------| +| `http/1.1`, `h2` | one-directional (client→server) | HTTP clients (curl, axios, `fetch` for request/response); SSE for subscription streaming (alkhttp ADR-049) | +| WebSocket (over `http/1.1` or `h2` upgrade) | **bidirectional** | Browser call-protocol clients; the path that restores the call protocol's bidirectionality for browsers | + +WebSocket is the surface that **restores the call protocol's +bidirectionality for browsers** (the role alknet ADR-043 §5 assigned to +WebTransport). The one-directional projection that alknet ADR-043 §5 +names for HTTP/1.1+HTTP/2 stands unchanged. (The `h3` row from the +original table does not exist in alkhttp — WebTransport is removed from +scope, alkhttp ADR-069.) + +### 5. Browsers over WebSocket are not peers — the rationale, stated. + +alkhttp ADR-034 §4 (ported from alknet ADR-034 §4) established that a +browser over WebTransport is not a peer (no `PeerId`, no +`PeerCompositeEnv` entry). The same applies to a browser over WebSocket, +and the rationale — which alknet ADR-034 §4 states as a closure without +the supporting argument — is worth making explicit because it is the +load-bearing distinction: + +**"Peer" means an addressable node in the call-protocol peer graph — a +stable `PeerId`, reachable via `PeerRef::Specific`, whose ops land in +`PeerCompositeEnv`, whose identity is stable across reconnects.** It +does *not* mean "any endpoint that exchanges calls during a live +session." A browser is the second thing but not the first, on three +concrete grounds: + +1. **No stable cryptographic identity of its own.** A `PeerEntry` is + anchored to fingerprints (Ed25519, X.509) that *the peer* presents + and the local node pins. A browser presents a bearer token the *hub* + issued; the "identity" is the hub's bookkeeping for that token, not + something the browser owns or that could be pinned by another node. + There is nothing to put in `PeerEntry.fingerprints`. +2. **Ephemeral.** Close the tab → connection dies → the connection-local + Layer 2 overlay (alkcall ADR-019) dies with it. A `PeerEntry` keyed + to a browser would be a permanently-dead entry within seconds. + `PeerRef::Specific("browser-X")` from another node would route to + nothing. +3. **Not addressable from other nodes.** `PeerRef::Specific` resolves + through `PeerEntry` → `PeerId`. Another node has no way to reach + "the browser currently connected to hub-A"; the hub holds that + connection as a live `CallConnection` handle, not as a peer-graph + entry. The connection-local overlay is precisely the mechanism that + gives the browser bidirectional-call capability *without* peer-graph + membership. + +This is the explicit closure of the "browser as peer" path, on both the +inbound (this section) and outbound (alkhttp ADR-034 §2) sides. The +browser is a **bidirectional call target during a live session**, not a +**peer-graph member**. The connection-local Layer 2 overlay (alkcall +ADR-019) is what makes the former possible without requiring the +latter. + +This rationale applies transport-agnostically — to WebSocket, to +WebTransport (an alknet transport, out of alkhttp scope per alkhttp +ADR-069), and to any future browser transport. alkhttp ADR-034 §4 is +amended by reference to this section. + +## Consequences + +**Positive:** +- alkhttp's first release does not carry a draft-standard protocol or + an experimental dependency stack on its security surface. The browser + path uses WebSocket, a mature, well-understood, RFC 6455 protocol with + first-class axum support (`axum::extract::ws`). +- The axum-bridge research spike for h3/WebTransport is not on the + critical path. WebSocket upgrade over HTTP/1.1 or HTTP/2 is standard + axum territory. +- The downstream crates that alkhttp unblocks (SSH, git, SFTP) are not + blocked on WebTransport or the proxy. They expose their ALPNs natively + over QUIC *(an alknet transport concern)*; browser reachability for + them would be a WebTransport feature — and in the alkhttp crate tree, + that is out of scope (alkhttp ADR-069), not a deferred feature. +- The crate stays lean: no `h3`, no `wtransport`/hyperium h3 stack, no + WebTransport feature gate. The only browser-bidirectional dependency + is the WebSocket upgrade path. + +**Negative:** +- alknet ADR-038, ADR-040, and ADR-043 are not implemented in this + crate and are not ported to it (see Status amendment). Their design + work is preserved in the alknet record only. A reader of alkhttp docs + must go to the alknet docs to find them; this ADR's status amendment + is the pointer. +- The ALPN-stream-proxy (alknet ADR-040) is not available anywhere in + the alkhttp surface. A browser cannot reach SSH/SFTP/git ALPNs + through this crate — it can reach the call protocol over WebSocket, + but not the non-call ALPNs. *(This is the alknet deferral; in + alkhttp it is a removal from scope, alkhttp ADR-069.)* +- WebSocket is a single stream; it lacks WebTransport's native + multi-stream multiplexing. For the call protocol this is fine + (correlation is by request ID, not by stream — alkcall ADR-015), and + WSS is the well-matched tool for that use case (see §"The tradeoff + between two use cases"). Where WebTransport's stream model would + matter is the ALPN-stream-proxy (alknet ADR-040) — the speculative + use case, which is out of alkhttp scope entirely (alkhttp ADR-069). +- The original ADR-044's "WebTransport restores bidirectionality" + framing (alknet ADR-043 §5) becomes "WebSocket restores + bidirectionality" — and in this crate that framing is final, not + provisional. + +## Reversal *(superseded by alkhttp ADR-069 for this crate)* + +The original decision reversed when a concrete deployment needed the +ALPN-stream-proxy — i.e., a real use case of a browser running a WASM +SSH/SFTP/git client to reach a non-call ALPN over WebTransport. At that +point, in the alknet record: + +1. The research spike deferred there (verify the hyperium stack's + server-side WebTransport API and the axum-bridge feasibility) is run. +2. alknet ADR-038 / ADR-040 / ADR-043 are un-parked and implemented as + written, with the `webtransport.md` spec as the design. +3. The WebSocket browser path (this ADR's §4) is not removed — it + remains as the simpler browser path for deployments that don't need + WebTransport's stream model. The two coexist. + +**In alkhttp this reversal path does not exist.** WebTransport is not +deferred here — it is removed from crate scope (alkhttp ADR-069). If a +WebTransport deployment is ever built, it is an alknet concern (a +transport/relay feature in the alknet tree); the alkhttp surface it +would front is the stable HTTP contract this crate publishes, and no +alkhttp ADR or feature gate anticipates it. The crate-surface one-way- +door reasoning in the original (an `h3` feature gate becoming part of a +published interface) is moot: alkhttp publishes no `h3` surface. + +## Research note (for revival) *(alknet concern; recorded for provenance only)* + +A note from the original record: `wtransport` (the reference +implementation read during initial research) is *probably not* the right +dependency choice, despite being a complete and readable implementation. +The load-bearing integration concern was that the `h3` handler must +route HTTP/3 requests through the same axum `Router` as `h2`/`http/1.1`, +and `wtransport` owns its own HTTP serving path — bridging its request +type into the `http::Request` axum consumes is cross-ecosystem adapter +work. The hyperium stack (`h3` + `h3-quinn` + `h3-webtransport`) +operates at the stream level and produces `http::Request` types +natively, which is a better fit for the axum integration — but its +server-side WebTransport API needs verification before commitment. + +This research was **not** run, and in alkhttp it never will be: there +is no WebTransport revival in this crate (alkhttp ADR-069). The note is +preserved only because the original record kept it for the alknet-side +revival question. + +## Assumptions + +1. **The call protocol's `EventEnvelope` framing fits the WebSocket + path cleanly.** In the original framing: an `EventEnvelope` is a + self-delimited JSON object; one envelope per WS binary message. In + the current wire model this is amended (see the framing correction + under Finding 1 and the alkhttp ADR-048 status amendment): the WS + message boundary carries channels chunks (8-byte chunk header, alkcall + ADR-034/ADR-035), and channel 0 — pre-negotiated as `alk/call` + (alkcall ADR-036) — carries `EventEnvelope` frames as length-prefixed + JSON (alkcall ADR-014's frame format) inside the chunk payload. The + load-bearing property — self-delimited frames, no streaming + deserializer across frame boundaries, correlation by request ID — is + unchanged. This is already verified by prior art: the + `@alkdev/pubsub` WebSocket client/server + (`/workspace/@alkdev/pubsub/src/event-target-websocket-client.ts`, + `event-target-websocket-server.ts`) carries the same + `{ type, id, payload }` envelope over WS binary messages; the call + protocol's `EventEnvelope` is a refined superset of that shape (typed + event names, structured payloads). + +2. **WebSocket upgrade over HTTP/1.1 or HTTP/2 is supported by the + axum/hyper stack natively.** `axum::extract::ws` provides the upgrade + handler; the underlying connection is the same hyper HTTP connection + the `h2`/`http/1.1` handler already drives. No new framing library is + needed. + +3. **A browser over WebSocket has the same peer-model properties as a + browser over WebTransport.** No `PeerId`, no `PeerCompositeEnv` + entry, connection-local Layer 2 overlay (alkcall ADR-019; alkhttp + ADR-034 §2). The rationale in §5 is transport-agnostic and applies + identically to WSS. + +4. **The downstream crates (SSH, git, SFTP) do not require WebTransport + or the ALPN-stream-proxy to exist.** They expose their ALPNs natively + over QUIC *(an alknet transport concern)*; the proxy is a browser- + reachability feature, not a prerequisite for the ALPNs themselves. + Browser reachability for non-call ALPNs is the speculative use case; + in alkhttp it is not deferred but out of scope (alkhttp ADR-069). + +## References + +- alkcall ADR-032 §"What this framework is NOT" — the anti-pattern + alknet ADR-038 was written to correct; the original decision relies + on the explicit distinction between deferral-as-hedging (rejected) + and deferral-as-scoping (permitted: a decision that "genuinely + doesn't need to be made yet because the use case isn't concrete" — + scope management, not door-type classification). *(Port note: the + one-way-door decision framework lives in the alkcall crate as + alkcall ADR-032 — see the alkcall crate docs; alkhttp did not port + it. The original linked alknet ADR-009 by relative path.)* +- alknet ADR-038 — **superseded by this ADR (in alknet).** Its + correction of the two-way-door-as-deferral anti-pattern stands; its + specific decision (h3 in scope now) is reversed. **Not ported to + alkhttp.** +- alknet ADR-040 — **parked, not superseded (alknet record).** Revives + unchanged when WebTransport revives as an alknet transport. The proxy + is the speculative use case. **Not ported to alkhttp.** +- alknet ADR-043 — **parked, not superseded (alknet record).** §2 + (bidirectionality) and §3 (no-`PeerId` overlay) transfer to WebSocket + unchanged; §4 (non-call-ALPN substrate) and §5's WebTransport-specific + framing revive with WebTransport as alknet work. **Not ported to + alkhttp.** +- alkhttp [ADR-034](034-outgoing-only-x509-and-three-peer-roles.md) §4 — + browsers are not peers; this ADR's §5 states the rationale + (addressability vs. bidirectionality) that the original §4 closes + without arguing. alkhttp ADR-034 §4 is amended by reference to this + ADR's §5. +- alkcall ADR-015 — call-protocol stream model; stream-agnostic + correlation, `Dispatcher`/`PendingRequestMap`; a WebSocket message + stream is another `BiStream`-satisfying transport. The call protocol + multiplexes by request ID, not by stream. +- alkcall ADR-014 — the call protocol's hand-rolled `EventEnvelope` + framing (length-prefixed JSON); the frame format now carried on + channel 0 of the WS path. +- alkcall ADR-034 / ADR-035 — the channels wire format (8-byte chunk + header); the multiplexing layer the WS path now carries (alkhttp + ADR-067). +- alkcall ADR-036 — channel 0 is pre-negotiated `alk/call`. +- alkhttp ADR-067 — the WS session carries the channels protocol; the + amendment that defines the current framing on this crate's WS path. +- alkhttp ADR-069 — WebTransport removed from alkhttp scope entirely. +- alkhttp ADR-049 — streaming handler for subscription operations (the + SSE projection for the HTTP one-directional path). +- alkhttp ADR-048 — the WS session shape (native session, not gateway); + the implementer-facing rule this ADR's §1 implied. +- alkhttp ADR-001 / ADR-002 — ALPN-based dispatch and the + `ProtocolHandler` trait (`HttpAdapter` on `h2`/`http/1.1`); the WS + upgrade layers on the same HTTP surface. +- The alkcall crate docs — the call-protocol spec and `EventEnvelope` + shape (see the alkcall crate's own documentation; the old + `call-protocol.md` relative link pointed into the alknet mono-repo + spec tree). + +## Port notes + +- **Superseded in part, per the status amendment above:** the "deferred" + framing is superseded by alkhttp ADR-069 (WebTransport removed from + alkhttp scope entirely). §1's WebSocket-as-browser-path stands; the + reversal trigger, the "revival" mechanics, the `webtransport.md` spec + pointer, and the research-spike-for-revival posture are alknet-record + content retained for provenance and annotated. alknet ADR-038/040/043 + are not ported to alkhttp. +- **Framing correction:** the original assumed "one `EventEnvelope` = one + WS binary message." In the current wire model the WS message boundary + carries 8-byte channels chunks (alkcall ADR-034/ADR-035); channel 0 is + pre-negotiated `alk/call` (alkcall ADR-036) and carries `EventEnvelope` + frames as length-prefixed JSON (alkcall ADR-014) inside the chunk + payload. The original sentences asserting one-envelope-per-WS-message + are annotated inline (Finding 1 correction, Assumption 1) rather than + silently rewritten; the operative statement for this crate is alkhttp + ADR-067. +- Renames: "alknet-http" → "alkhttp"; the QUIC call ALPN "alknet/call" → + "alk/call" (alkcall ADR-004 `alk/` convention); "alknet ADR-012" (the + old call-protocol stream model) is cited as **alkcall ADR-015** + (alkcall ADR numbering differs from alknet numbering — alkcall ADR-012 + is `ConnectionCredentials`, not the stream model). +- The original's references to `crates/http/webtransport.md` and + `crates/http/http-server.md` are alknet spec-tree artifacts; the + `webtransport.md` spec does not exist in alkhttp (no deferred + WebTransport spec is carried in this crate). The alkhttp equivalent of + the http-server spec work is the `websocket`/`server` subsystem of + this crate and alkhttp ADR-048. +- The `@alkdev/pubsub` prior-art references are kept as absolute + workspace paths (the packages still exist at those locations); the + framing delta note is annotated per the channels-chunk correction. +- The original table in §4 had an implied `h3` row (WebTransport → + alk/http in the alknet ALPN table); that row is dropped from the + alkhttp transport table per alkhttp ADR-069, with a note in §4. +- iroh-relay precedent: kept — it is a design-signal argument, not a + dependency claim; alkhttp has no iroh dependency. +- Original title preserved: "Defer h3/WebTransport; Browsers Use + WebSocket". The title's "defer" is the alknet-record posture; in + alkhttp the operative reading is "no WebTransport in this crate; + browsers use WebSocket" (alkhttp ADR-069). \ No newline at end of file diff --git a/docs/architecture/decisions/045-to-openapi-gateway-spec-versioning.md b/docs/architecture/decisions/045-to-openapi-gateway-spec-versioning.md new file mode 100644 index 0000000..d197c97 --- /dev/null +++ b/docs/architecture/decisions/045-to-openapi-gateway-spec-versioning.md @@ -0,0 +1,231 @@ +# ADR-045: to_openapi Gateway-Spec Versioning + +*Ported from alknet ADR-045 (to_openapi Gateway-Spec Versioning); re-targeted to alkhttp.* + +## Status + +Proposed + +## Context + +OQ-39 asked how the published `to_openapi` spec is versioned. [ADR-017](017-call-protocol-client-and-adapter-contract.md) +Consequences established that a published `to_*` spec is a compatibility +contract: once external clients build against it, the mapping semantics +become a de facto contract and changing them breaks every client. + +The original framing of OQ-39 assumed `to_openapi` generated a +traditional per-operation-paths OpenAPI doc — one path per `External` +operation, changing whenever an operation is added, removed, or has its +schema modified. Under that model the versioning surface is large and +churns constantly, and the doc is a static full-surface dump (the Gitea +failure mode: admin ops shown to every caller, no per-caller filtering). + +[ADR-042](042-openapi-gateway-pattern.md) replaced that model with the +**gateway pattern**: `to_openapi` +generates a doc describing **5 fixed gateway endpoints** +(`/search`, `/schema`, `/call`, `/batch`, `/subscribe`), and the +per-caller operation surface is discovered at runtime through +`AccessControl`-filtered `/search` — not preloaded into the static doc. +This is the same mechanic as the MCP gateway (ADR-041), with `subscribe` +added because OpenAPI/SSE supports streaming where MCP tool calls are +request/response. + +The consequence for versioning: the published doc is now a small, stable +surface that changes only when the gateway endpoint set or an endpoint's +request/response shape changes. Per-caller operation changes +(adding/removing/modifying operations, changing an operation's schema) +do **not** change the published doc — those operations are not in the +doc; they are discovered via `/search`. This dissolves most of the +churn the original OQ-39 was concerned about. + +What remains is the narrow versioning question: how does the published +gateway doc signal its version so consumers can detect breaking changes? +This is one-way after first publication — once external clients build +against the gateway doc, renaming `/call` or changing its request shape +breaks them. + +A note on door-type framing: ADR-009 classifies doors by reversal cost +in the codebase. The "published artifact is a contract" case is a blind +spot in that framework — the published doc's reversal cost is paid by +external consumers, not in the codebase. [ADR-017](017-call-protocol-client-and-adapter-contract.md) +Consequences captures +this (published `to_*` specs are compatibility contracts); this ADR +honors the constraint without changing ADR-009's framework. The door is +two-way before first publication (the gateway shape can be revised +freely while no external client depends on it) and one-way after +(revising requires a major version bump that signals breakage to +consumers). + +## Decision + +### 1. The published gateway doc carries a semver `info.version` + +`to_openapi` emits `info.version` as a semver string. The version +reflects the **gateway endpoint contract** (the 5 endpoints + their +request/response shapes), not the operation set: + +- **Major bump** — breaking change to the gateway contract: an endpoint + removed or renamed, a required field added to a gateway endpoint's + request, a response shape changed in a backward-incompatible way + (including removing or retyping an existing response field, or + tightening an optional field to required), + the error-mapping semantics ([ADR-023](023-operation-error-schemas.md)) changed. +- **Minor bump** — additive change: a new gateway endpoint added + (e.g., a future `/subscribe-batch`), a new optional request field, a + new response field. Additive changes do not break existing clients. +- **Patch bump** — description/wording changes, documentation, no shape + change. + +Cases not enumerated above follow **standard semver**: a change is a +major bump if it could break a client built against the prior version, +a minor bump if it is purely additive, a patch bump otherwise. The +enumerated triggers above are the common cases, not an exhaustive list. + +Per-caller operation changes (registering a new operation, removing one, +changing an operation's input schema) **do not bump the version** — the +operation set is not part of the published doc; it is discovered via +`/search` at runtime. This is the key simplification the gateway pattern +buys: the operation surface can evolve freely without touching the +published contract version. + +**alkhttp note:** the gateway contract here is the 5-endpoint set, +extended with `/publish` in alkhttp (ADR-068). That addition is a minor +bump under this ADR's rules (a new gateway endpoint added, additive — +existing clients are unaffected; the versioning rationale is unchanged), +and the contract this ADR governs in alkhttp is the resulting 6-endpoint +set. + +### 2. The version is bumped on change to the gateway shape, not on regeneration + +A deployment that regenerates the doc (e.g., on restart) gets the same +`info.version` unless the gateway shape changed. The version is a +function of the gateway contract, not of when the doc was generated. + +### 3. Consumers detect breaking changes via the major version + +A client reading the doc compares `info.version`'s major component to +the version it built against. A major bump signals "re-read the doc, +something broke." The minor/patch components are informational. This is +the standard OpenAPI/semver convention — no alkhttp-specific detection +mechanism. + +### 4. The traditional per-operation-paths projection (additive, ADR-042 §5) versions independently + +A deployment that builds the additive traditional REST projection +(ADR-042 §5) versions that doc on its own schedule — its surface +*does* change with the operation set, so its versioning is the +per-operation churn OQ-39 originally worried about. That projection is +opt-in and out of scope for this ADR; the gateway doc is the default +published contract and the one this ADR governs. + +## Consequences + +**Positive:** +- The published contract is a 5-endpoint surface (6 in alkhttp with + `/publish`, ADR-068) that rarely changes. + Versioning is bump-on-change, not bump-on-every-operation-change. The + original OQ-39 concern (constant churn) is dissolved by the gateway + pattern — the operation set is not in the doc. +- Consumers use standard semver/OpenAPI `info.version` — no + alkhttp-specific version-detection mechanism to learn. +- Per-caller operation evolution (the common case) is decoupled from the + published-contract version. A node can add/remove operations freely + without bumping the doc version or breaking clients built against the + gateway doc. +- The Gitea failure mode stays structurally impossible (ADR-042 §3): + `/search` is `AccessControl`-filtered, so the doc never exposes ops + the caller can't call. Versioning inherits this — the doc describes + the gateway, not the operations. + +**Negative:** +- A client cannot tell from the doc version alone *which* operations are + available — it must call `/search`. This is by design (per-caller, + runtime), but a client expecting a static operation list from the doc + must learn the gateway pattern. +- The version only signals gateway-contract changes. An operation + changing its input schema (a breaking change for callers of that + operation) does not bump the doc version — that change is surfaced via + `/schema` per-operation, not via the doc version. Clients that cache + operation schemas must re-fetch `/schema` to detect per-operation + changes; the doc version does not track them. + +## Assumptions + +1. **The 5-endpoint gateway set is stable.** ADR-042 Assumption 1. Adding + endpoints is additive (minor bump); removing/renaming is a major bump. + The initial 5-endpoint set is the first published contract (alkhttp + extends it with `/publish`, ADR-068 — an additive minor-bump + addition). + +2. **Per-operation schema changes are detected via `/schema`, not the + doc version.** The doc version tracks the gateway contract only. A + client that caches an operation's `OperationSpec` re-fetches `/schema` + to detect changes to that operation. This is the standard + discovery-then-invoke pattern; the doc version is not a per-operation + change tracker. + +3. **`info.version` is the single source of truth for the published + contract version.** No separate `x-alknet-version` extension or + content-hash header. Standard OpenAPI field, standard semver + interpretation. A content-hash would be more precise but adds an + alkhttp-specific mechanism for no real gain over semver-on-shape- + change. + +## References + +- alknet ADR-009: One-Way Door Decision Framework (now alkcall ADR-032) + — door-type + framework (classifies by codebase reversal cost; the + published-artifact-as-contract case is the blind spot this ADR honors + without changing the framework) +- [ADR-017](017-call-protocol-client-and-adapter-contract.md) — published + `to_*` specs are compatibility contracts (the one-way-after- + publication constraint; the decision record is alkcall ADR-022) +- [ADR-023](023-operation-error-schemas.md) — error-mapping semantics + are part of the gateway contract (a change to them is a major bump; + the decision record is alkcall ADR-016) +- [ADR-036](036-http-to-call-operation-mapping.md) — the SSE projection + for `/subscribe` (part of the gateway contract) +- [ADR-042](042-openapi-gateway-pattern.md) — the gateway pattern that + makes the published doc a 5-endpoint surface instead of a per- + operation surface; §4 explicitly deferred versioning to OQ-39 +- [ADR-068](068-gateway-publish-endpoint.md) — the alkhttp `/publish` + gateway endpoint; its addition bumps the gateway contract version as + a minor bump under this ADR's rules +- alknet OQ-39 — `to_openapi` published-spec versioning (resolved by + this ADR) +- `http-adapters.md` (in this crate's `docs/architecture/`) — the spec + that emits `info.version` (alknet original: + `crates/http/http-adapters.md`) + +## Port notes + +- Renames: "alknet" → alkhttp where the system/node being described is + meant (including the original's "no alknet-specific detection + mechanism", "no `x-alknet-version` extension", "an alknet-specific + mechanism" — now alkhttp-specific / `x-alkhttp-version` semantics); + the protocol-foundation crate is alkcall. +- **Gateway endpoint count / version bump**: the original decision text + (5 endpoints) is preserved verbatim as history; alkhttp's `/publish` + addition (ADR-068) is noted as an explicit alkhttp note in Decision §1 + and in the References — per this ADR's own rules it is an additive + minor bump of the gateway contract; the versioning rationale is + unchanged. +- Producer/consumer terminology: the original body contained no + call-protocol server/client role framing; the client roles mentioned + (clients built against the doc, clients caching schemas) are + HTTP/OpenAPI inherent-directionality roles and keep their names. +- Cross-reference remappings (verified alknet→alkcall ADR mapping): + alknet ADR-009 (one-way door framework) → alkcall ADR-032, cited + textually. ADR-017/023 are ported to this crate under the same + numbers and linked; their alkcall record numbers are noted alongside + (017→alkcall ADR-022, 023→alkcall ADR-016). +- OQ-39 is an alknet OQ-tracker item, cited textually; it is resolved by + this ADR, which this crate ports under the same number. +- Spec-document links converted per alkhttp conventions: + `crates/http/http-adapters.md` → `http-adapters.md` in this crate's + `docs/architecture/`. +- No decision content changed — the semver rules, bump-on-shape-change + rule, major-version detection, the additive-projection carve-out, + consequences, and assumptions are verbatim from the alknet ADR modulo + the adaptations above. \ No newline at end of file diff --git a/docs/architecture/decisions/046-assembly-layer-custom-http-routes.md b/docs/architecture/decisions/046-assembly-layer-custom-http-routes.md new file mode 100644 index 0000000..84f3a06 --- /dev/null +++ b/docs/architecture/decisions/046-assembly-layer-custom-http-routes.md @@ -0,0 +1,268 @@ +# ADR-046: Assembly-Layer Custom HTTP Routes on HttpAdapter + +*Ported from alknet ADR-046 (Assembly-Layer Custom HTTP Routes on HttpAdapter); re-targeted to alkhttp.* + +## Status + +Proposed + +## Context + +The `HttpAdapter` (see `http-server.md` in `docs/architecture/`) is +constructed by the assembly layer with an `Arc`, an +`Arc`, and a `DecoyConfig`. The axum `Router` it +builds has a fixed surface: + +- The `to_openapi` gateway endpoints (`/search`, `/schema`, `/call`, + `/batch`, `/subscribe` — [ADR-042](042-openapi-gateway-pattern.md)) — the sole invoke path over HTTP + ([ADR-047](047-remove-direct-call-http-surface.md) removed the direct-call `POST /{service}/{op}` surface that + ADR-036 originally defined). +- `/healthz`, `/openapi.json`, the MCP route (feature-gated), and the + decoy fallback for unknown paths. + +There is no documented extension point for a downstream deployment to +add its own HTTP routes to this router. A deployment that wants to +expose a custom HTTP endpoint — one that is *not* a gateway endpoint — +has no specified way to do so. The architecture currently ties the HTTP +surface to the simplified contract with no escape hatch. + +### The concrete use case + +A hub deployment (e.g., `api.alk.dev`) wants to expose the standard +alknet contract (direct-call + gateway) **and** an OpenAI-compatible +proxy at `/v1/chat/completions`. The OAI proxy is a custom HTTP route: +it receives an OpenAI-shaped request, dispatches into the +`OperationRegistry` (likely to a `from_openapi`-imported `openai/chat` +operation or a custom agent operation), and returns an OpenAI-shaped +response. It is not an alknet operation — it is a deployment-specific +HTTP endpoint that uses the registry as a backend. + +This pattern is not exotic. It is the standard "wrap an external API +shape around our operations" pattern: a deployment adds a +compatibility shim (OAI-compatible, Anthropic-compatible, a legacy API +shape) as a custom route, backed by call-protocol operations. The +alternative — forcing every custom endpoint to be a call-protocol +operation whose input/output match the external API's shape — is +brittle (the OAI streaming response shape is not a clean call-protocol +output) and unnecessary (the deployment owns the HTTP shape; the +registry owns the operation shape). + +The runner pattern that motivates this (remote GPU instance downloads +a binary, connects back to the hub via `from_call`, registers its ops; +opencode connects to the hub as a standard OAI provider) is already +supported by the existing architecture (`from_call`, `PeerRef` routing, +`from_openapi` to wrap the OAI API). The only missing piece is the +custom HTTP route on the hub. + +### Why this needs an ADR + +The extension mechanism — how the assembly layer injects custom routes +— is a published API surface of `HttpAdapter`. Once downstream +deployments build against it (passing their custom routers at +construction), changing the mechanism is a one-way door (every consumer +construction site breaks). It needs an ADR before implementation so the +contract is deliberate, not accidental. + +The specific routes a deployment adds are a two-way door (add/remove +freely, no protocol contract). The *mechanism* (the constructor +parameter and its semantics) is the one-way door this ADR commits. + +## Decision + +### 1. HttpAdapter accepts additional axum routes from the assembly layer + +The `HttpAdapter` constructor gains a parameter for deployment-specific +routes. The assembly layer builds an `axum::Router` with its custom +routes and passes it in; `HttpAdapter` composes it with the default +surface. A deployment that passes no custom routes gets exactly the +documented default behavior — the extension point is additive, not +mandatory. + +```rust +pub struct HttpAdapter { + identity_provider: Arc, + registry: Arc, + decoy: DecoyConfig, + /// Deployment-specific routes added by the assembly layer. None = + /// the default surface only. See ADR-046. + extra_routes: Option, +} +``` + +The exact composition mechanism (merge vs nest vs builder, whether +custom routes get a prefix) is a two-way-door implementation detail; +the one-way constraint is that the assembly layer can inject routes +and they coexist with the default surface. axum's `Router::merge` / +`Router::nest` are the natural primitives. + +### 2. Custom routes are raw HTTP, not call-protocol operations + +A custom route is a raw axum handler — it receives an HTTP request and +returns an HTTP response. It is not registered in the +`OperationRegistry`, not discoverable via `/search`, not described in +the `to_openapi` gateway doc. The deployment owns its shape entirely. + +A custom route *may* dispatch into the `OperationRegistry` (via +`OperationRegistry::invoke()`, same as the gateway's `/call` endpoint +does) if +it wants to back the HTTP endpoint with a call-protocol operation. The +OAI-compatible proxy does this: the `/v1/chat/completions` handler +parses the OAI request, invokes the `openai/chat` (or `agent/chat`) +operation, and reformats the response as an OAI response. But this is +the custom route's choice — it could equally be a pure HTTP handler +that never touches the registry (a webhook receiver, a static asset +server, a legacy API shim with its own backend). + +### 3. The default surface's reserved paths take precedence on collision + +The default-surface paths are reserved: `/search`, `/schema`, `/call`, +`/batch`, `/subscribe`, `/healthz`, `/openapi.json`, and the MCP route. +(ADR-047 removed the direct-call `/{service}/{op}` surface, so it is no +longer a reserved path; a deployment that builds a per-operation +projection as a custom route is the one case where `/{service}/{op}` +patterns appear, and those custom routes are subject to the same +collision rule.) If a custom route collides with a reserved path, the +default surface wins — the custom route is silently shadowed (or the +construction panics/warns; the specific collision-handling is a +two-way-door implementation detail). A deployment that wants +`/v1/chat/completions` namespaces it away from the reserved set, which +is natural (`/v1/...` doesn't collide). + +### 4. Custom routes carry the same auth middleware by default; per-route opt-out is the deployment's choice + +Custom routes run under the same Bearer-auth resolution as the default +surface (the `Authorization: Bearer` → `resolve_from_token` path). A +deployment that wants a custom route to be unauthenticated (a public +webhook receiver, a health endpoint with a different shape than +`/healthz`) applies axum middleware to opt that route out of auth — +the deployment owns its custom routes' middleware stack. The +`HttpAdapter` provides the identity provider and the default auth +middleware; the custom `Router` the assembly layer passes in can +layer its own middleware on top. This is standard axum composition; no +alknet-specific mechanism. + +### 5. Custom routes are not part of the published `to_openapi` doc + +The `to_openapi` gateway doc (ADR-042, ADR-045) describes the 5 +gateway endpoints — the default contract. Custom routes are +deployment-specific and not described by `to_openapi`. A deployment +that wants its custom routes documented for external consumers +generates its own OpenAPI doc for them (a separate projection, not +`to_openapi`). The default `info.version` semver (ADR-045) tracks the +gateway contract, not custom routes — custom routes have no +versioning contract with alknet; the deployment versions them however +it wants. + +### 6. This does not change the default surface + +A deployment that constructs `HttpAdapter` with no extra routes gets +exactly the behavior documented in `http-server.md` — gateway, +`/healthz`, `/openapi.json`, MCP (feature-gated), decoy. The +extension point is purely additive. The default surface remains the +published contract (ADR-042, ADR-045; ADR-036's routing decision is +superseded by ADR-047); custom routes are a +deployment-specific addition on top, not a modification of it. + +## Consequences + +**Positive:** +- Deployments can wrap external API shapes (OAI-compatible, + Anthropic-compatible, legacy) around call-protocol operations without + forcing the external shape into the operation's input/output schema. + The "compatibility shim" pattern is first-class. +- The runner pattern (remote worker → `from_call` → hub → custom OAI + route → opencode) works end-to-end with no architectural gap. The hub + is a standard alknet node *plus* a deployment-specific HTTP surface. +- The extension point is standard axum composition — no alknet-specific + routing abstraction for deployers to learn. A developer who knows + axum can add routes. +- The default surface is unchanged for deployments that don't need + custom routes. No complexity tax for the common case. + +**Negative:** +- The HTTP surface is no longer fully described by the alknet specs + alone — a deployment's custom routes are outside the architecture + docs. This is inherent to the extension point (the deployment owns + them); the specs describe the *default* surface and the *mechanism*, + not every possible custom route. +- A custom route that dispatches into the registry bypasses the + gateway's `AccessControl`-filtered `/search` discovery — the custom + route is responsible for its own authorization story. The default + Bearer-auth middleware covers the common case, but a custom route + that wants per-operation ACL checks must call + `OperationRegistry::invoke()` with a proper `OperationContext` + (caller identity from the resolved bearer token), not a bypass. The + `invoke()` path enforces `AccessControl` regardless of the entry + point (direct-call, gateway, or custom route), so this is not an + ACL bypass — but the custom route author must construct the context + correctly. +- Two deployments with custom routes have different HTTP surfaces — + there is no single "what does an alknet HTTP endpoint look like" + answer anymore. The default surface is the contract; custom routes + are deployment-specific variance. This is honest (deployments + *do* vary) but means the architecture docs describe the default, not + the union. + +## Assumptions + +1. **The assembly layer is the composition point.** Custom routes are + added at `HttpAdapter` construction, not registered dynamically at + runtime. This matches the static-registration constraint (OQ-04 / + [ADR-010](010-alpn-router-and-endpoint.md)) for the + `HandlerRegistry`; the `HttpAdapter`'s router is + likewise immutable after construction. Dynamic route addition would + require `ArcSwap` and is not part of this ADR. + +2. **Custom routes are a deployment concern, not an alknet-crate + concern.** alkhttp provides the extension point (accepts the + extra `Router`); it does not provide custom route implementations. + The OAI-compatible proxy, the legacy API shim, the webhook receiver + are all written by the deployment (or a downstream crate like + alknet-agent that builds on alkhttp), not by alkhttp + itself. + +3. **The default surface is the published contract; custom routes are + not.** ADR-036 (direct-call), ADR-042 (gateway), ADR-045 + (versioning) govern the default surface. Custom routes have no + alknet-governed compatibility contract — the deployment owns their + stability. This keeps the published-contract surface small and + stable while allowing arbitrary deployment-specific extension. + +4. **axum's composition primitives are sufficient.** `Router::merge`, + `Router::nest`, and axum middleware cover the extension patterns + needed (custom routes, per-route auth opt-out, prefix namespacing). + No alknet-specific routing abstraction is required. If a future + need exceeds axum's composition (e.g., route-level dynamic dispatch), + that would be a separate ADR. + +## References + +- [ADR-010](010-alpn-router-and-endpoint.md) — static registration at + startup (the `HttpAdapter` router is immutable after construction, + same constraint) +- [ADR-042](042-openapi-gateway-pattern.md) — the gateway endpoints + (the default surface custom routes coexist with; reserved paths) +- [ADR-045](045-to-openapi-gateway-spec-versioning.md) — the published + doc versions the gateway contract, not custom routes +- [ADR-047](047-remove-direct-call-http-surface.md) — the direct-call + surface is removed; the gateway is the sole invoke path (a + deployment that wants the former per-operation HTTP surface builds it + as a custom route projection; this ADR §4 is the mechanism) +- `http-server.md` — the `HttpAdapter` spec that gains the + `extra_routes` constructor parameter (now in `docs/architecture/`) + +## Port notes + +- Link path fixes: `crates/http/http-server.md` → `http-server.md` in + `docs/architecture/` (referenced both in the Context and in the + References). Cross-ADR links (ADR-010, ADR-042, ADR-045, ADR-047) + point at `decisions/NNN-.md` with the alknet slugs; ADR-042, + ADR-045, and ADR-047 are being ported by other agents under the same + numbers and will exist. ADR-036 is referenced by number/text only + (its mapping decision is superseded by ADR-047). +- `resolve_from_token` on a Bearer header is the `IdentityProvider` + path defined in ADR-004 (ported here under the same number); + `IdentityProvider` internals are alkcall-internal. +- No decision content changed — the `extra_routes` mechanism, reserved + paths, auth middleware, and versioning rules are verbatim from the + alknet ADR. \ No newline at end of file diff --git a/docs/architecture/decisions/047-remove-direct-call-http-surface.md b/docs/architecture/decisions/047-remove-direct-call-http-surface.md new file mode 100644 index 0000000..4501204 --- /dev/null +++ b/docs/architecture/decisions/047-remove-direct-call-http-surface.md @@ -0,0 +1,351 @@ +# ADR-047: Remove the Direct-Call HTTP Surface; Gateway Is the Sole Invoke Path + +*Ported from alknet ADR-047 (Remove the Direct-Call HTTP Surface; Gateway Is the Sole Invoke Path); re-targeted to alkhttp.* + +## Status + +Proposed + +## Supersedes + +The "direct path mapping" clause of +[ADR-036](decisions/036-http-to-call-operation-mapping.md) +§Decision ("Direct path mapping is the default HTTP surface") and §HTTP +method semantics. ADR-036's other clauses (SSE projection, Bearer auth, +`/healthz`, stealth decoy, error mapping, `External`-only dispatch) +remain in force — they are independent of the routing decision and are +reaffirmed by this ADR (see §"What survives from ADR-036"). + +## Context + +ADR-036 defined the HTTP surface as **direct path mapping**: +`POST /{service}/{op}` → `call.requested` for every `External` +operation. An operation `fs/readFile` was served at `POST /fs/readFile`, +one HTTP path per operation — a REST-like surface mirroring the call +protocol's `/{service}/{op}` operation paths. This was the original HTTP +contract, decided before the simplified-contract / gateway-pattern +work landed. + +Since then, three shifts made the direct-call surface a contradiction +with the architecture's settled model: + +1. **ADR-042** replaced `to_openapi`'s per-operation-paths projection + with the **gateway pattern** — 5 fixed endpoints (`/search`, + `/schema`, `/call`, `/batch`, `/subscribe`) where the per-caller + operation surface is discovered via `AccessControl`-filtered + `/search`, not preloaded into a static doc. The gateway's `/call` + endpoint is the invoke path: `POST /call` with + `{ operation: "/fs/readFile", input: {...} }`. This is the same + RPC-shape pattern MCP uses (`tools/call` with a tool name, + [ADR-041](041-mcp-tool-gateway-pattern.md)). + +2. **The simplified contract is the few-fixed-endpoints model**, not a + per-operation REST tree. The whole point of the gateway pattern + (ADR-042) was to escape the "static full-surface dump" failure mode + (the Gitea anti-pattern: every operation gets a path, every caller + sees the full surface, per-caller access is an afterthought). The + direct-call surface is that anti-pattern at the HTTP level: every + `External` operation gets an HTTP path, the path exists regardless + of the caller's privilege, and the caller discovers what it can call + by trial-and-error `403`s. The gateway's `/search` exists precisely + to make the per-caller surface the default; the direct-call surface + re-introduces the problem the gateway solved. + +3. **ADR-046** added the custom-routes extension point, so a + deployment that genuinely wants a REST-like per-operation HTTP + surface (e.g., to match a legacy API shape) builds it as a custom + route projection (additive, deployment-owned, not the alkhttp + default contract). The direct-call surface is no longer the only + way to get per-operation HTTP paths; it's the *default* way, and + it's the wrong default. + +The result: the HTTP router currently has **two ways to invoke an +operation** — the direct-call surface (`POST /fs/readFile`) and the +gateway (`POST /call` with the operation name in the body). That is the +contradiction: the simplified contract says "a few core endpoints," +and the direct-call surface is a second, per-operation invoke path that +duplicates the gateway's `/call` with a scheme the gateway was built +to replace. ADR-042's amendment explicitly preserved the direct-call +surface ("unchanged"); that preservation was a leftover from before +the simplified contract was fully thought through, not a deliberate +endorsement of two invoke paths. + +### The clean-up + +The direct-call surface is residual from early-stage planning, the +same way the pre-ADR-042 `to_openapi` per-operation-paths projection +was residual. ADR-042 cleaned up `to_openapi`; this ADR cleans up the +HTTP handler's routing. The gateway becomes the sole invoke path; the +per-operation HTTP paths go away. + +### What about HTTP clients that knew operation names? + +A client that previously called `POST /fs/readFile` now calls +`POST /call` with `{ "operation": "/fs/readFile", "input": {...} }`. The +operation name is still the call protocol's `/{service}/{op}` form +(OQ-13, unchanged) — it moves from the HTTP path to the request body. +The gateway's `/call` is the standard invoke endpoint; the direct path +was a REST-like affordance that the simplified contract deliberately +drops. This is a breaking change for any HTTP client built against the +direct-call surface, which is exactly why it needs an ADR — but the +direct-call surface has not been implemented or published yet (the +alkhttp crate is specced, not shipped), so the "break" is +paper-only: no external client depends on it. + +## Decision + +> **Extension note (alkhttp):** the gateway this ADR narrows the HTTP +> surface to now has **six** endpoints, not the five listed in the +> original decision. alkhttp ADR-068 +> (`decisions/068-gateway-publish-endpoint.md`) added `/publish` (POST; +> invokes Pub operations — producer→consumer streaming; request body +> streamed as newline-delimited JSON, each line one published chunk; +> final `ResponseEnvelope` returned as the HTTP response) to the +> gateway contract after this ADR was decided. "The 5 gateway +> endpoints" below is the original decision text, retained as decision +> history; the sole-invoke-path decision applies to the 6-endpoint +> gateway (`/search`, `/schema`, `/call`, `/batch`, `/subscribe`, +> `/publish`). + +### 1. The gateway is the sole invoke path; the direct-call surface is removed + +The `HttpAdapter`'s router serves the **5 fixed gateway endpoints** +(`/search`, `/schema`, `/call`, `/batch`, `/subscribe` — ADR-042) as +the only way to invoke operations over HTTP. There is no +`POST /{service}/{op}` direct-call surface. An HTTP client invokes an +operation by `POST /call` with +`{ "operation": "/{service}/{op}", "input": {...} }`. + +The router's operation-invoke surface is the gateway's `/call` +endpoint, not a per-operation path set. The operation name is in the +request body, not the HTTP path — same shape as MCP's `tools/call` +([ADR-041](041-mcp-tool-gateway-pattern.md)) and the call protocol's +own `call.requested` +(`operationId` + `input`). + +### 2. The HTTP method semantics move to the gateway endpoints + +ADR-036's `OperationType` → HTTP method mapping (`Query`→`GET`, +`Mutation`→`POST`, `Sub`→`SSE`) no longer applies per-operation +at the HTTP path level, because there are no per-operation HTTP paths. +The gateway endpoints have fixed methods (ADR-042's table): +`/search` `GET`, `/schema` `GET`, `/call` `POST`, `/batch` `POST`, +`/subscribe` `POST` (SSE). The `OperationType` of the *called operation* +is carried in the request/result, not expressed in the HTTP verb — the +client calls `/call` with the operation name; the operation's type is +the registry's concern, not the HTTP method's. A `Query` operation and a +`Mutation` operation both go through `POST /call`; the distinction is +in the operation spec (discovered via `/schema`), not the HTTP surface. + +### 3. What survives from ADR-036 + +ADR-036's routing decision is superseded, but its other clauses are +independent of routing and remain in force: + +- **SSE projection for subscriptions over `h2`/`http/1.1`** (§Streaming + projection). The gateway's `/subscribe` endpoint uses this SSE + projection (ADR-042 §2). The framing (`call.responded` → SSE `data:` + frame, `call.completed` → stream close, `call.aborted` → error frame) + is unchanged; it is now the `/subscribe` endpoint's behavior, not a + per-operation SSE stream. +- **Bearer auth** (§Auth). `Authorization: Bearer` → + `resolve_from_token` on every gateway endpoint. Unchanged. +- **`/healthz`** (§`/healthz` and operational endpoints). Raw route, no + auth, no call protocol. Unchanged. +- **Stealth decoy** (§Stealth mode). Unknown paths get the decoy. + Unchanged — and now *all* operation invocations go through the 5 + gateway paths, so the "unknown path" surface is larger (anything not + `/search`, `/schema`, `/call`, `/batch`, `/subscribe`, `/healthz`, + `/openapi.json`, the MCP route, or a custom route per ADR-046 is + decoy). +- **Error mapping** (the call `code` → HTTP status table in + http-server.md, per the alkcall crate's ADR-016). The gateway's + `/call` endpoint returns the same error mapping. Unchanged in + mechanism; the entry point is `/call` instead of `/{service}/{op}`. +- **`External`-only dispatch** (Assumption 2). The gateway's `/call` + returns `404` (`NOT_FOUND`) for `Internal` operations, same as the + direct-call surface did. The `AccessControl` check runs on the called + operation regardless of the entry point. +- **Abort cascade on HTTP disconnect** (Consequences, citing alkcall + ADR-020). An HTTP consumer disconnecting mid-`/subscribe` is detected + as a stream close and sends `call.aborted`, cascading to descendants. + Unchanged. + +### 4. A deployment that wants per-operation HTTP paths builds them as custom routes (ADR-046) + +A deployment that genuinely needs a REST-like per-operation HTTP +surface (to match a legacy API shape, to serve clients that can't +adapt to the gateway) builds it as a **custom route projection** +([ADR-046](046-assembly-layer-custom-http-routes.md)): the assembly +layer injects an `axum::Router` with +`POST /{service}/{op}` handlers that dispatch into +`OperationRegistry::invoke()`. This is deployment-owned, additive, and +explicitly *not* the alkhttp default contract — the same status as an +OAI-compatible proxy. The direct-call surface is no longer a built-in +default; it's a projection a deployment can build if it needs it, on +the same extension point as any other custom HTTP surface. + +This keeps the default surface small (5 gateway endpoints) while +preserving the *capability* for REST-like access — it just isn't free +by default, which is correct, because the per-operation path surface +has real costs (the static-surface problem) that the gateway avoids. + +### 5. `to_openapi` describes the gateway, unchanged + +`to_openapi` (ADR-042, +[ADR-045](045-to-openapi-gateway-spec-versioning.md)) already describes +the 5 gateway +endpoints, not per-operation paths. Removing the direct-call surface +does not change what `to_openapi` generates — it already generated the +gateway doc. The `info.version` semver (ADR-045) tracks the gateway +contract; the direct-call surface was never in that contract. No change +to `to_openapi` or its versioning. + +## Consequences + +**Positive:** +- One invoke path over HTTP, not two. The HTTP surface is the 5 gateway + endpoints — exactly the "few core endpoints" of the simplified + contract. The contradiction with the gateway pattern is resolved. +- The per-caller API surface is the default, structurally. An HTTP + client cannot stub its toe on `POST /admin/deleteUser` because that + path does not exist; it calls `/call` with the operation name, and + `/search` tells it what it can call. The Gitea failure mode is + structurally impossible at the HTTP level, not just at the discovery + level. +- The HTTP surface is honest about what the call protocol is: an RPC, + not a REST API. The gateway's `/call` with `{ operation, input }` is + the call protocol's own shape; the direct path mapping was a REST + disguise that didn't fit (the flat JSON input, no path/query/body + split — ADR-042 §"The flat→structured problem"). +- A deployment that wants REST-like per-operation paths still can, via + custom routes (ADR-046) — it's an explicit choice with its own costs, + not a default that leaks the static-surface problem into every + deployment. +- No change to `to_openapi` (already described the gateway), to the + SSE projection (now on `/subscribe`), to Bearer auth, to `/healthz`, + to stealth, or to error mapping. The cleanup is narrow: the routing + decision only. + +**Negative:** +- An HTTP client that knew an operation name can no longer call it at + a predictable HTTP path. It must call `/call` with the operation name + in the body. This is one layer of indirection, but it's the same + indirection MCP uses and the same shape the call protocol uses + natively. The operation name (OQ-13's `/{service}/{op}` form) is + unchanged — it moves from the path to the body. +- The HTTP surface is RPC-shaped, not REST-shaped. A developer + expecting `POST /fs/readFile` sees `POST /call` with a body instead. + This is honest (the call protocol is a flat JSON RPC, ADR-042 §3), but + it's a departure from the REST conventions ADR-036's direct-call + surface offered. A deployment that needs the REST shape builds it as a + custom route projection (ADR-046). +- The `OperationType` → HTTP method mapping (`Query`→`GET` etc.) no + longer applies at the HTTP level. A `Query` operation and a + `Mutation` operation both go through `POST /call`. The distinction is + in the operation spec (visible via `/schema`), not the HTTP verb. This + loses a small amount of HTTP-level signal (a load balancer can't tell + a read from a write by method), but the call protocol's + `OperationType` was always a registry concern, not an HTTP concern — + the direct-call surface borrowed HTTP verbs to express it, and the + gateway doesn't. + +## Assumptions + +1. **No external client depends on the direct-call surface.** The + alkhttp crate is specced, not shipped; the direct-call surface + has not been published. Removing it is a paper-only break — no + deployed client breaks. This is why the cleanup is cheap now and + would be expensive after implementation. + +2. **The gateway's `/call` is a sufficient invoke path for HTTP + clients.** Any operation callable via `POST /{service}/{op}` is + callable via `POST /call` with the operation name in the body. The + operation name form (`/{service}/{op}`, OQ-13) is unchanged. The + input/output shapes are unchanged. The only difference is where the + operation name lives (path vs body). + +3. **A deployment needing REST-like per-operation paths builds them + explicitly.** Via ADR-046 custom routes. This is not a common need — + the gateway's `/call` covers the standard invoke case, and the + OAI-compatible-proxy pattern (ADR-046) covers the "match an external + API shape" case. The direct-call surface was a default that served + neither case particularly well (it wasn't REST-conventional, per + ADR-036 §Negative, and it leaked the static-surface problem). + +4. **The gateway endpoints are stable (ADR-042 Assumption 1).** + Removing the direct-call surface does not change the gateway + endpoint set; the 5 endpoints are the published contract. This ADR + narrows the HTTP surface *to* that contract, it does not modify the + contract itself. + +## References + +- [ADR-036](decisions/036-http-to-call-operation-mapping.md) — the ADR + whose + routing decision this supersedes (§Decision, §HTTP method semantics); + its other clauses survive (§"What survives from ADR-036") +- [ADR-042](decisions/042-openapi-gateway-pattern.md) — the gateway + pattern that + made the direct-call surface redundant; its amendment to ADR-036 + preserved the direct-call surface, which this ADR reverses +- [ADR-044](044-defer-webtransport-browsers-use-websocket.md) — + WebSocket is the browser bidirectional path (the direct-call surface + was the `h2`/`http/1.1` one-directional path; removing it does not + affect WebSocket, which carries the call protocol natively) +- [ADR-046](046-assembly-layer-custom-http-routes.md) — the extension + point a deployment uses to build a per-operation HTTP surface if it + needs one (the direct-call surface's replacement for the rare case) +- [ADR-045](045-to-openapi-gateway-spec-versioning.md) — `to_openapi` + versions the gateway contract (unchanged; the direct-call surface + was never in the contract) +- alkhttp ADR-068 (`decisions/068-gateway-publish-endpoint.md`) — adds + `/publish` to the gateway contract (the 6th endpoint; see the + extension note above) +- OQ-13 (resolved) — operation path format `/{service}/{op}` is + unchanged; it moves from the HTTP path to the `/call` request body +- `docs/architecture/http-server.md` — this crate's spec whose router + surface this ADR narrows to the gateway endpoints + +## Port notes + +- Renames: "alknet-http" → alkhttp throughout (crate name, shipped-vs- + specced status, threat-model and default-contract references). +- **Gateway endpoint count (extension note, never silently rewritten):** + the original decision text says "the 5 fixed gateway endpoints" + (`/search`, `/schema`, `/call`, `/batch`, `/subscribe`) throughout. + In alkhttp, alkcall ADR-046 (`Pub` operation type) motivated a sixth + gateway endpoint, `/publish` (producer→consumer streaming; newline- + delimited JSON request body; final `ResponseEnvelope` as the HTTP + response), recorded in this crate's ADR-068. A block-quote extension + note at the top of §Decision states the 6-endpoint contract and marks + the "5 endpoints" phrasing below it as retained decision history; + the original text is otherwise verbatim. The stealth-decoy clause §3 + retains the original 5-path enumeration for the same reason. +- **`Subscription` → `Sub`** (§2's method-mapping sentence; alkcall + rename of the `OperationType` value). +- **Producer/consumer terminology**: §3 abort-cascade bullet "An HTTP + client disconnecting mid-`/subscribe`" → "An HTTP consumer + disconnecting mid-`/subscribe`" (call-protocol role framing; the + abort-cascade citation is alkcall ADR-020). HTTP server/client + phrasing elsewhere is HTTP-inherent and untouched (`HttpAdapter` + serves; HTTP clients call). +- **Cross-refs**: `../../decisions/...` → `decisions/...`. ADR-036 and + ADR-042 are ported to this crate under their alknet numbers/slugs and + linked (`decisions/036-http-to-call-operation-mapping.md`, + `decisions/042-openapi-gateway-pattern.md`); ADR-044/045/046 are + ported under the same numbers and linked directly. The source's + alknet ADR-023 (error schemas) citation is remapped textually to the + alkcall crate's ADR-016 (`decisions/016-operation-error-schemas.md`); + the alknet ADR-016 (abort cascade) citation is remapped to alkcall + ADR-020. The new-ADR link `decisions/068-gateway-publish-endpoint.md` + uses the alkhttp slug for ADR-068 (the alknet ADR-068 is an unrelated + hub decision and is not referenced here). +- `crates/http/http-server.md` → `docs/architecture/http-server.md` + (this crate's docs/architecture/). +- No frontmatter in the source; status kept as Proposed. +- No decision content changed — the sole-invoke-path decision, the + method-semantics move, the "what survives" list, the custom-routes + escape hatch, the `to_openapi` unchanged clause, and the + consequences/assumptions are verbatim from the alknet ADR modulo the + renames and the logged six-endpoint extension note. \ No newline at end of file diff --git a/docs/architecture/decisions/048-websocket-native-session-not-gateway.md b/docs/architecture/decisions/048-websocket-native-session-not-gateway.md new file mode 100644 index 0000000..3e0712e --- /dev/null +++ b/docs/architecture/decisions/048-websocket-native-session-not-gateway.md @@ -0,0 +1,408 @@ +# ADR-048: WebSocket Carries the Native Call-Protocol Session, Not the Gateway Shape + +*Ported from alknet ADR-048 (WebSocket Carries the Native Call-Protocol Session, Not the Gateway Shape); re-targeted to alkhttp.* + +## Status + +Accepted + +## Status amendment (alkhttp port) + +- **The WS path carries the channels session** (alkhttp ADR-067): a + browser WS connection is demultiplexed by the channels protocol — + 8-byte chunk headers per alkcall ADR-034/ADR-035 — rather than being a + bare `EventEnvelope` stream. **Channel 0 is pre-negotiated as + `alk/call`** (alkcall ADR-036) and carries the native call-protocol + session **exactly as this ADR describes** — the dispatch loop, the + overlay rules, and the browsers-not-peers property all stand + unchanged; they apply to channel 0. +- **Framing on channel 0 is length-prefixed JSON** (alkcall's frame + format, alkcall ADR-014) **inside the 8-byte chunk header** — **not** + one-envelope-per-WS-message. The WS message boundary carries chunks, + not envelopes; see alkhttp ADR-067 §framing. Sentences below that + speak of "one `EventEnvelope` = one binary WS message" are the + original 2026 framing, retained as decision history; the operative + framing is the channels-chunk model just described. +- **The default WS upgrade path is `/alk/channels`** (was + `/alknet/call` in the alknet original — renamed per the alkcall ADR-004 + `alk/` ALPN convention and per alkhttp ADR-067's channels session). + +## Context + +alkhttp ADR-044 (Accepted) removed h3/WebTransport from this crate's +scope (see alkhttp ADR-069; the alknet record deferred it) and committed +WebSocket as the browser bidirectional path: a browser upgrades an +HTTP/1.1 or HTTP/2 request to WebSocket and the resulting full-duplex WS +connection carries the call protocol's `EventEnvelope` frames (in the +current wire model: on channel 0, as length-prefixed JSON inside +channels chunks). alkhttp ADR-044 §1 established that the call +protocol's `call.requested`/`call.responded`/`call.completed`/`call.aborted` +exchange "works over WebSocket with no protocol change — the same `Dispatcher`, +the same `PendingRequestMap`, the same correlation by request ID." + +alkhttp ADR-044 §1 also established *what shape the WS session carries* — the native +`EventEnvelope` call-protocol session — but it does so as part of a larger +argument about why WebTransport isn't required, not as a crisp rule an +implementer is told not to violate. Two facts make the distinction worth its +own explicit decision record: + +1. **The HTTP surface has a deliberate, well-documented invoke contract: the + `to_openapi` gateway pattern** (alkhttp ADR-042, alkhttp ADR-047). The gateway is 5 fixed + endpoints (`/search`, `/schema`, `/call`, `/batch`, `/subscribe`), where + `/call` takes `{ "operation": "/fs/readFile", "input": {...} }` and invokes + through `OperationRegistry::invoke()`. It is a well-shaped, simple contract. + An implementer writing the WS handler could plausibly ask: "should the WS + path expose the same 5-endpoint gateway shape, so a WS client looks like an + HTTP client?" That is a reasonable question, and the answer is no — but the + answer is currently implicit in alkhttp ADR-044's framing, not stated as a rule. + +2. **The two surfaces serve different architectural roles, and that difference + is load-bearing.** The HTTP gateway is, by HTTP's nature, a *one-directional + projection* — client initiates, server responds (see the alkhttp server + spec, §"One-directional projection"). The whole reason WebSocket exists in + this architecture is to *restore the call protocol's native bidirectionality + for browsers* (alkhttp ADR-044 §4): a WS connection is full-duplex, so both + sides can initiate `call.requested` frames. Putting the gateway's + one-directional shape on WS re-introduces the one-directional limitation + that WS exists to fix. The two surfaces are not interchangeable; they are + deliberately different tools for deliberately different jobs. + +### The two invoke contracts, contrasted + +| Aspect | HTTP gateway (`/call`, alkhttp ADR-042/047) | WS native session (this ADR) | +|---------|--------------------------------------|------------------------------| +| Direction | One-directional (client→server calls only) | Bidirectional (either side can `call.requested`) | +| Wire unit | HTTP request/response | Channels chunk (8-byte header, alkcall ADR-034/035); channel 0 payload = length-prefixed-JSON `EventEnvelope` (alkcall ADR-014) | +| Invoke shape | `POST /call` with `{ "operation": "/fs/readFile", "input": {...} }` | `call.requested` event with `{ operation, input }` payload (the call protocol's native shape) | +| Discovery | `GET /search` (gateway endpoint) | `services/list` as an ordinary call-protocol op | +| Schema | `GET /schema` (gateway endpoint) | `services/schema` as an ordinary call-protocol op | +| Streaming | `POST /subscribe` (SSE frames) | `call.responded` events as channel-0 frames (no SSE) | +| Dispatcher | axum route handler → `OperationRegistry::invoke()` | shared `Dispatcher` (alkcall ADR-015, stream-agnostic) | +| Multiplexing | HTTP/2 native; HTTP/1.1 sequential | Channels channel IDs (alkcall ADR-034/035) + request ID (alkcall ADR-015) | + +The WS row is the call protocol's own native session, with WebSocket as the +transport instead of QUIC (carried on channel 0 of the channels session per +alkcall ADR-036). The HTTP row is a projection of that session into +HTTP's one-directional shape, with the gateway as the deliberate interface for +clients that only speak HTTP. + +### Why the gateway shape is wrong on WS + +Three concrete reasons: + +1. **It duplicates the native invoke path with a lossier one.** The call + protocol's `call.requested` event *is* the invoke primitive; the gateway's + `/call` is that primitive wrapped in an HTTP envelope. On WS, the envelope + is unnecessary — there is no HTTP request/response cycle to fit into. A + gateway-on-WS would be a translation layer translating the call protocol to + itself, losing bidirectionality in the process. + +2. **It loses the per-caller filtering property the native session already + has.** The gateway's `/search` exists to give HTTP clients the + `AccessControl::check(identity)`-filtered discovery that the call protocol + provides natively via `services/list`. On WS, `services/list` is already a + call-protocol op the browser can call directly — the filtering is already + there. A gateway-on-WS re-implements a filtering property the native session + already provides. + +3. **It breaks the symmetry with the QUIC path.** The `alk/call` QUIC path + (alkcall ADR-015, alkcall ADR-022) is the native `EventEnvelope` session over + QUIC bidirectional streams. WS is the same session over WS messages (via + the channels session's channel 0 — alkcall ADR-036). Making the + WS path different from the QUIC path (by putting the gateway on WS) creates + two browser-reachable invoke contracts for no architectural reason — the + QUIC path is the reference, and WS should mirror it, not diverge from it. + +### The prior art is already native-session-shaped + +The `@alkdev/pubsub` WebSocket client/server (`event-target-websocket-client.ts`, +`event-target-websocket-server.ts`) — the working prior art alkhttp ADR-044 cites as +the reason the WS path is cheap — already carries the `EventEnvelope { type, id, +payload }` shape over WS binary messages, with no gateway-style wrapping. The +call protocol's `EventEnvelope` was derived from the pubsub envelope +(refined with typed event names and structured payloads); the delta is small +and well-defined (see the alkcall crate docs, §"Transport agnosticism"). +A browser/Node WS client derived from the pubsub +prior art speaks the native session shape, not a gateway shape. The +gateway-on-WS variant would require *un-translating* the pubsub client's +native-session shape into the gateway's `{ operation, input }` shape — work +that has no payoff because the native shape is what both the QUIC path and the +pubsub prior art use. *(In the current wire model, the client additionally +speaks the channels chunk framing around those envelopes — alkcall ADR-034/035 +via alkhttp ADR-067.)* + +## Decision + +### 1. A WebSocket connection is a native `EventEnvelope` call-protocol session, not the HTTP gateway shape + +The WS handler on `HttpAdapter` hands the WS message stream to the call +protocol's shared `Dispatcher` — the same dispatch loop the call adapter +uses for `alk/call` QUIC connections (alkcall ADR-015, stream-agnostic +correlation; a WS message stream is another `BiStream`-satisfying +transport). Concretely in alkhttp (per alkhttp ADR-067): the WS handler +demultiplexes the 8-byte chunk framing of the channels protocol (alkcall +ADR-034/ADR-035); **channel 0 is pre-negotiated as `alk/call`** +(alkcall ADR-036) and its chunk payloads are `EventEnvelope` frames in +alkcall's length-prefixed JSON frame format (alkcall ADR-014). The +browser writes `EventEnvelope` frames as channel-0 chunks; the handler +reads them and dispatches via `OperationRegistry::invoke()`. Responses +(`call.responded`, `call.error`, `call.completed`, `call.aborted`) are +written back as channel-0 chunks. + +The `to_openapi` gateway endpoints (`/search`, `/schema`, `/call`, `/batch`, +`/subscribe` — alkhttp ADR-042, alkhttp ADR-047) **do not appear on the +WebSocket path**. They are the HTTP one-directional projection's invoke +contract; WS carries the call protocol's own native session, which is a +different (and richer) thing. + +### 2. Discovery and schema are call-protocol ops, not WS-specific endpoints + +The browser calls `services/list` and `services/schema` as ordinary +`call.requested` events over the WS connection (channel 0). They are +call-protocol operations, not WS endpoints. There is no `/search` or +`/schema` on WS — those are the HTTP gateway's names for the same +discovery primitives. The filtering the gateway provides via +`AccessControl::check(identity)`-filtered `/search` (alkhttp ADR-042 §3) +is provided on the WS path by the same mechanism the call protocol uses +everywhere: `services/list` is `AccessControl`-filtered natively (see +the alkcall crate docs, client-and-adapters, §"services/list"). No +WS-specific discovery surface exists or is needed. + +### 3. Subscriptions project as native `call.responded` events, not SSE + +A `Subscription` operation invoked over WS streams `call.responded` +events as channel-0 frames directly — no SSE `data:` framing (that is +the `h2`/`http/1.1` projection for `/subscribe`, per the streaming +handler decision, alkhttp ADR-049; on WS it is unnecessary because WS is +already a framed full-duplex channel). `call.completed` closes the +stream; `call.aborted` closes it with an error frame. This is the +native streaming projection for the WS path, mirroring how subscriptions +work on the QUIC path. + +### 4. Bidirectionality is native and unchanged from the QUIC path + +The WS call-protocol session inherits the call protocol's native +bidirectionality (alknet ADR-043 §2, transferred to WebSocket per +alkhttp ADR-044 §3): both sides can send `call.requested` frames. The +browser calls operations on the hub; the hub can call operations +registered on the browser's side, over the same session, using the same +`PendingRequestMap` and `EventEnvelope` framing as `alk/call`. The +browser case where the client registers no operations of its own is the +common case — the server→client call direction is unused because the +browser has nothing to call. That is a use-case scoping, not an +architectural limitation. + +### 5. This is a clarifying decision, not a new one + +alkhttp ADR-044 §1 commits the native-session shape ("the call +protocol's framing fits the WebSocket path cleanly ... the same +`Dispatcher`, the same `PendingRequestMap`, the same correlation by +request ID"). This ADR does not change that decision; it makes the +implication an explicit, implementer-visible rule: **the WS path is the +native session, and the gateway shape is deliberately not applied to +it.** An implementer reading alkhttp ADR-044 alone could plausibly ask +"should the WS path expose the gateway endpoints too?" — this ADR's job +is to make the answer discoverable as a decision record, not implicit in +framing. + +## Consequences + +**Positive:** +- One invoke model for the call protocol, regardless of transport. The + QUIC path and the WS path run the same `EventEnvelope` session through + the same `Dispatcher` (on the WS path, via channel 0 of the channels + session — alkhttp ADR-067); the HTTP gateway is the one-directional + projection for clients that only speak HTTP. An implementer building + the WS handler reuses the `Dispatcher` and `OperationRegistry::invoke()` + dispatch path verbatim — no WS-specific routing, no WS-specific + discovery surface, no second invoke contract to design or maintain. +- The `@alkdev/pubsub`/`@alkdev/operations` TypeScript clients sync to + the call protocol with no translation layer: their `EventEnvelope` + shape is already the native session shape, and the call protocol's + envelope is a refined superset of the pubsub envelope (alkhttp ADR-044 + §"Concrete prior art"). The gateway-on-WS variant would have required + un-translating the pubsub client's native-session shape; this decision + avoids that un-translation entirely. *(The clients also speak the + channels chunk layer around those envelopes — alkcall ADR-034/035.)* +- Per-caller `AccessControl`-filtered discovery is already a property of + the native session (`services/list`). No WS-specific filtering surface + to build or document; the call protocol's authorization model applies + unchanged. +- The browser is a bidirectional call target during a live session, not + a peer-graph member (alkhttp ADR-044 §5, alkhttp ADR-034 §4). The + native session shape is what makes this clean: the browser gets + bidirectional call capability through the connection-local Layer 2 + overlay (alkcall ADR-019) without peer-graph membership, and the + gateway shape would not have changed this — but it would have made the + WS path diverge from the QUIC path for no benefit. + +**Negative:** +- A WS client cannot use the gateway's `{ "operation": ..., "input": ... }` + body shape — it must speak the call protocol's native `call.requested` + event (inside the channels chunk framing). This is honest (the WS path + *is* the call protocol), but a developer who learned the gateway shape + from the HTTP surface must learn the `EventEnvelope` shape (plus the + chunk framing) for WS. The pubsub prior art and the + `@alkdev/operations` TypeScript client already speak the envelope + shape, so the delta is small for the primary consumer — but it is a + real difference from the HTTP gateway's simpler `{ operation, input }` + invoke body. +- The 5 gateway endpoint names (`/search`, `/schema`, `/call`, `/batch`, + `/subscribe`) are HTTP-specific and do not carry over to WS. A deployment + documenting its surface for both HTTP and WS clients documents two invoke + shapes (the gateway for HTTP; the native session for WS). This is the cost + of using the right tool for each transport instead of forcing one shape onto + both. + +## Reversal + +This ADR clarifies a decision alkhttp ADR-044 already committed (§1 +describes the native session; this ADR makes that the explicit, +implementer-visible rule). The reversal posture is therefore alkhttp +ADR-044's, not a separate one: the WS path itself is not deferred (it is +the browser path), and the native-session-not-gateway choice is a +clarification of what that path carries — reversing it would mean +adopting the gateway shape on WS, which would re-introduce the +one-directional limitation WS exists to fix (§Context reason 1). The +original text's realistic reversal path — "WebTransport revives and adds +a second browser bidirectional path" — is an alknet posture and does not +apply to this crate: WebTransport is removed from alkhttp scope entirely +(alkhttp ADR-069), and the ALPN-stream-proxy (alknet ADR-040) is an +alknet record, not ported here. If a second browser bidirectional +transport ever exists, it is an alknet transport concern fronting the +stable HTTP surface this crate publishes; this ADR's rule (WS = native +session on the channels path, not gateway) is unaffected — the gateway +shape stays HTTP-only regardless of how many browser bidirectional +transports exist. + +## Assumptions + +1. **The call protocol's `EventEnvelope` framing fits the WebSocket path + cleanly.** In the original 2026 framing: an `EventEnvelope` is a + self-delimited JSON object; one envelope per WS binary message. In + the current wire model (alkhttp ADR-067): the WS message boundary + carries channels chunks (8-byte header, alkcall ADR-034/ADR-035), and + channel 0 — pre-negotiated as `alk/call` (alkcall ADR-036) — carries + `EventEnvelope` frames as length-prefixed JSON (alkcall ADR-014's + frame format) inside the chunk payload. The load-bearing property — + self-delimited frames, no streaming deserializer across frame + boundaries — is unchanged. This is verified by prior art: the + `@alkdev/pubsub` WebSocket client/server carries the same + `{ type, id, payload }` envelope over WS binary messages. + +2. **The shared `Dispatcher` runs over the WS path unchanged.** alkcall + ADR-015 commits stream-agnostic correlation; a WS message stream is + another `BiStream`-satisfying transport (per alkcall ADR-038, the + channels `ChannelConnection` is itself a `BiStreamSource`). The + `Dispatcher` and `PendingRequestMap` are transport-agnostic; only the + connection-establishment half differs (WS upgrade handler vs QUIC + accept/dial). + +3. **The primary WS consumer is a browser or Node client derived from + the `@alkdev/pubsub`/`@alkdev/operations` prior art.** That client + already speaks the native `EventEnvelope` shape (now wrapped in the + channels chunk framing). The gateway's simpler `{ operation, input }` + body shape is the HTTP path's affordance for clients that only speak + HTTP; a client that has chosen WS has already opted into the call + protocol's native framing. + +4. **`services/list` and `services/schema` are sufficient discovery for + the WS path.** They are `AccessControl`-filtered (per-caller) and + return the full `OperationSpec` respectively. The gateway's `/search` + and `/schema` are HTTP-shaped names for these same primitives; on WS + the primitives apply directly. No WS-specific discovery surface is + needed. + +## References + +- alkhttp ADR-067 — **the amendment that defines the current WS wire + model**: the WS session carries the channels protocol (8-byte chunk + multiplexing); channel 0 is pre-negotiated `alk/call` and carries the + native call-protocol session described here. +- alkcall ADR-034 / alkcall ADR-035 — the channels wire format (8-byte + chunk header; pure channel multiplexing). +- alkcall ADR-036 — channel 0 is pre-negotiated `alk/call`. +- alkcall ADR-014 — the call protocol's hand-rolled `EventEnvelope` + framing (length-prefixed JSON); the frame format carried in channel-0 + chunk payloads. +- alkcall ADR-015 — call-protocol stream model; stream-agnostic + correlation (`Dispatcher`/`PendingRequestMap`); a WS message stream is + another `BiStream`-satisfying transport. +- alkcall ADR-038 — `ChannelConnection` as a `BiStreamSource`; the + channels-session side of the stream-agnostic claim. +- alkcall ADR-022 §5 — `to_*` adapters are projections that consume the + registry; WS is not a `to_*` adapter (it carries the native session, + it doesn't project it). +- alkcall ADR-019 — Layer 2 per-connection overlay where + browser-registered ops (if any) land. +- alkhttp [ADR-034](034-outgoing-only-x509-and-three-peer-roles.md) §4 + (amended by alkhttp ADR-044 §5) — browsers are not peers; the + connection-local overlay gives the browser bidirectional-call + capability without peer-graph membership. +- alkhttp ADR-049 — the SSE projection for `/subscribe` (the HTTP + one-directional streaming path; on WS, subscriptions project as native + `call.responded` events, no SSE). +- alkhttp ADR-042 — the gateway pattern this ADR clarifies is HTTP-only. +- alknet ADR-043 §2/§3 — bidirectionality and the no-`PeerId` + connection-local overlay, transferred to WebSocket per alkhttp ADR-044 + §3 *(alknet record; not ported to alkhttp)*. +- alkhttp [ADR-044](044-defer-webtransport-browsers-use-websocket.md) — + the ADR that committed WS as the browser path; this ADR clarifies the + shape of what it committed (§1 implies the native session; this ADR + makes it an explicit rule). +- alkhttp ADR-047 — the gateway as the sole HTTP invoke path (the + HTTP-only contract this ADR clarifies does not extend to WS). +- alkhttp ADR-001 / ADR-002 — ALPN-based dispatch; `HttpAdapter` as the + `ProtocolHandler` for `h2`/`http/1.1`; the WS upgrade rides the HTTP + surface. +- alkhttp ADR-069 — WebTransport removed from alkhttp scope (context for + the reversal posture above). +- The alkcall crate docs — call-protocol spec (§"Transport + agnosticism") and client-and-adapters spec (§"services/list"); the old + relative links into the alknet spec tree became textual references to + the alkcall crate's own documentation. + +## Port notes + +- **Amended by alkhttp ADR-067, per the status amendment above:** the WS + session now carries the **channels protocol** (8-byte chunk + multiplexing, alkcall ADR-034/035) with **channel 0 pre-negotiated as + `alk/call`** (alkcall ADR-036) instead of a bare `EventEnvelope` WS + message stream. The dispatch/overlay/browsers-not-peers content of + this ADR stands unchanged and applies to channel 0. The framing on + channel 0 is **length-prefixed JSON** (alkcall ADR-014) inside the + chunk payload — **not one-envelope-per-WS-message**; the WS message + boundary carries chunks, not envelopes (alkhttp ADR-067 §framing). + Original one-envelope-per-message sentences are retained as decision + history with inline annotations (Decision 1, §Context table, + Assumptions 1). +- **Upgrade path rename:** the default WS upgrade path is `/alk/channels` + (was `/alknet/call`) — renamed per the alkcall ADR-004 `alk/` ALPN + convention and per alkhttp ADR-067 (the path carries the channels + session). The original statement "/alknet/call" appears nowhere in the + ported body except as this note. +- Renames: "alknet-http" → "alkhttp"; `CallAdapter` (old name in the + dispatch-loop sentence) is the call-protocol adapter now vendored in + the alkcall crate; "alknet ADR-012" (stream model) → **alkcall + ADR-015**; "alknet ADR-017 §5" (`to_*` projection) → **alkcall ADR-022 + §5**; "alknet ADR-024" (registry layering) → **alkcall ADR-019**. + alkcall ADR numbers differ from alknet ADR numbers; each citation + names the owning crate. +- alknet ADR-036 (SSE mapping) and alknet ADR-049 (streaming handler) + correspond to **alkhttp ADR-049** in this crate; the SSE `/subscribe` + references are re-cited accordingly. +- alknet ADR-043 is cited as an alknet record (not ported to alkhttp); + its §2/§3 transfers are restated with alkcall ADR-019 as the overlay + citation. +- Links rewritten per alkhttp conventions: `../crates/http/...` and + `../crates/call/...` relative links became textual "alkhttp server + spec" / "alkcall crate docs" references (the original + `http-server.md` §"WebSocket browser path" and `websocket.md` spec + pointers describe alknet spec-tree documents; in this crate the + equivalent content lives in the `server`/`websocket` subsystem specs + to be written under `docs/architecture/`). +- `OperationRegistry::invoke()`, `Dispatcher`, `CallConnection`, + `PendingRequestMap`, `EventEnvelope` are alkcall-owned types (see + alkcall ADR-014/015/019/022); cited textually, not re-defined here. +- Original title preserved: "WebSocket Carries the Native Call-Protocol + Session, Not the Gateway Shape". \ No newline at end of file diff --git a/docs/architecture/decisions/049-streaming-handler-for-subscriptions.md b/docs/architecture/decisions/049-streaming-handler-for-subscriptions.md new file mode 100644 index 0000000..fdfb393 --- /dev/null +++ b/docs/architecture/decisions/049-streaming-handler-for-subscriptions.md @@ -0,0 +1,422 @@ +# ADR-049: Streaming Handler for Subscription Operations + +*Ported from alknet ADR-049 (Streaming Handler for Subscription Operations); re-targeted to alkhttp.* + +## Status + +Accepted + +## Context + +The call protocol defines `Sub` as a first-class operation type +(alkcall ADR-015 lists `subscribe` as one of four top-level protocol +operations; `OperationSpec.op_type` includes `Sub`). The wire protocol supports +streaming: five event types (`call.requested`, `call.responded`, +`call.completed`, `call.aborted`, `call.error`), `PendingRequestMap::Subscribe` +with an mpsc channel, `CallConnection::subscribe()` returning +`impl Stream`, and a full streaming-subscribe example +in the alkcall crate's `call-protocol.md`. The **consumer side** works — a +consumer can subscribe to a remote stream and consume `call.responded` events +until `call.completed`. + +The **producer side did not (in the original design).** The `Handler` type in +the call crate was: + +```rust +pub type Handler = Arc< + dyn Fn(Value, OperationContext) -> Pin + Send>> + + Send + Sync, +>; +``` + +It returns a single `ResponseEnvelope`. `OperationRegistry::invoke()` returns +one `ResponseEnvelope` and closes. `Dispatcher::handle_stream` calls +`dispatch_requested` → `registry.invoke()` → writes one `EventEnvelope` frame → +loops. A `Sub` operation that should produce a *stream* of +`call.responded` events followed by `call.completed` has no way to express that +through this handler signature. + +This is a **spec gap that should not have shipped.** The TypeScript predecessor +(`@alkdev/operations`, from which the Rust port was derived) had two distinct +handler types: + +```typescript +type OperationHandler = (input: I, context: C) => Promise | O; +type SubscriptionHandler = (input: I, context: C) => AsyncGenerator; +``` + +The TS registry (`registry.ts:21`) stored them as a union, validated at +registration that `SUBSCRIPTION` ops get an `AsyncGeneratorFunction`, and the +dispatch (`call.ts:341-349`, `buildCallHandler`) branched on +`op_type`: `SUBSCRIPTION` → iterate the async generator, `respond()` for each, +then `complete()`; else → `execute()`, single `respond()`. The Rust port +collapsed the union into a single `Handler` returning one `ResponseEnvelope`, +losing the streaming path. The fix is to restore it. + +The downstream consequences of the gap: + +- **`/subscribe` HTTP endpoint** (`GatewayDispatch::invoke()` → + `subscribe_handler`) wraps a single `ResponseEnvelope` in a one-event SSE + stream. A real `Sub` operation (e.g., `agent/chat` streaming LLM + tokens) cannot stream through it. +- **`from_call` forwarding** for a `Sub` op calls + `CallConnection::call_with_payload()` (single response), not + `CallConnection::subscribe()` (stream). A `from_call`-imported subscription + truncates to the first value. +- **`from_openapi` forwarding** for a `text/event-stream` response returns one + `ResponseEnvelope` instead of streaming the SSE chunks. + +All three are symptoms of the same root cause: the `Handler` type cannot +produce a stream. + +## Decision + +### 1. `StreamingHandler` type (alongside `Handler`) + +Add a streaming handler type that returns a stream of `ResponseEnvelope`s, +mirroring the TS `SubscriptionHandler` / `OperationHandler` split: + +```rust +pub type StreamingHandler = Arc< + dyn Fn(Value, OperationContext) + -> Pin + Send>> + + Send + Sync, +>; +``` + +Each `Ok(value)` in the stream becomes a `call.responded` event. An `Err` +becomes a `call.error` event (terminal — the stream ends). Natural stream end +becomes `call.completed`. The dispatch path converts each `ResponseEnvelope` to +`EventEnvelope` exactly as it does today for the single-response case — no new +wire-format concept is introduced. + +A `make_streaming_handler()` helper (analogue of `make_handler()`) wraps an +async generator / stream-producing closure into a `StreamingHandler`. + +### 2. `HandlerKind` enum on `HandlerRegistration` + +```rust +pub enum HandlerKind { + Once(Handler), + Stream(StreamingHandler), + Sink(SinkHandler), +} + +pub struct HandlerRegistration { + pub spec: OperationSpec, + pub handler: HandlerKind, // validated against spec.op_type at registration + pub provenance: OperationProvenance, + pub composition_authority: Option, + pub scoped_env: Option, + pub capabilities: Capabilities, +} +``` + +Registration validates: `Query` / `Mutation` → `HandlerKind::Once`; +`Sub` → `HandlerKind::Stream`; `Pub` → `HandlerKind::Sink`. Mismatch is a +startup error (same as +the TS `validateSubscriptionHandler`). The enum makes the "one or the other, +matching `op_type`" invariant type-level rather than two `Option`s validated at +runtime. + +> **`Sink` variant (alkcall ADR-046).** The original decision defined two +> variants, `Once` and `Stream`, covering the four operation types then in +> existence (`Query`, `Mutation`, `Sub`) — `Sub` ops → `Stream`, everything +> else → `Once`. alkcall ADR-046 subsequently added `OperationType::Pub` +> (producer→consumer streaming via the `call.published` wire event) with +> `HandlerKind::Sink` as the third variant. The `Sink` variant shown above is +> annotated into the enum for accuracy of the current model; the original +> two-variant decision text below is retained as decision history. In v1, +> `from_openapi` produces no `Pub` ops (SSE responses detect as `Sub`), so +> this ADR's adapter discussion is unaffected by `Sink`. + +### 3. `OperationRegistry::invoke_streaming()` + +```rust +impl OperationRegistry { + /// Dispatch a Sub operation. Returns a stream of + /// ResponseEnvelopes. Errors (not-found, forbidden, invalid operation + /// type) yield a single ResponseEnvelope::error and end the stream. + pub fn invoke_streaming( + &self, + name: &str, + input: Value, + context: OperationContext, + ) -> BoxStream; +} +``` + +`invoke_streaming()` performs the same visibility + ACL checks as `invoke()`, +then dispatches to the `StreamingHandler`. Pre-handler errors (not-found, +forbidden) produce a single error `ResponseEnvelope` and end the stream +(matching the single-response path's behavior, just on a stream). + +### 4. `OperationRegistry::invoke()` errors on `Sub` + +`invoke()` is the request/response dispatch path. Calling it on a +`Sub` op is a type mismatch — a streaming operation dispatched through +the request/response path. It returns a `ResponseEnvelope` carrying +`CallError { code: "INVALID_OPERATION_TYPE", ... }` (a new protocol-level +code): + +``` +INVALID_OPERATION_TYPE +``` + +(`retryable: false`, `details: None`). This is the wire-format addition: a +sixth protocol-level error code. It signals "you called the wrong dispatch +method for this operation's type" — distinct from `INVALID_INPUT` (schema +mismatch) and `INTERNAL` (handler failure). Consumers should treat unknown codes +as `INTERNAL` with `retryable: false` (the existing rule); `INVALID_OPERATION_ +TYPE` is a permanent caller-side programming error, not a transient failure. + +### 5. `OperationEnv::invoke()` errors on `Sub` + +`OperationEnv::invoke()` (composition) stays request/response-only. It returns +a single `ResponseEnvelope`. Calling it on a `Sub` op produces the +same `INVALID_OPERATION_TYPE` error — composition cannot truncate a stream to +its first value. This is a clean architectural boundary, not a deferral: + +- **`OperationEnv` composition** is "call a child operation, get a result" + (the `OperationHandler` model). It is request/response by construction. +- **Stream composition** (filter, map, combine, window, dedupe) is a + handler-level concern. A handler that produces a stream transforms it with + stream operators at the handler level, not through `OperationEnv`. The + `@alkdev/pubsub` `operators.ts` is the prior art for this model: 13 operators + (`filter`, `map`, `take`, `batch`, `dedupe`, `window`, `chain`, `join`, etc.) + that operate on `AsyncIterable`, distinct from the request/response + composition. In Rust, the analogues operate on `BoxStream`. +- No `invoke_streaming()` is added to `OperationEnv`. The protocol composition + surface is request/response; stream manipulation is handler-internal. + +### 6. Dispatch branches on `op_type` + +`Dispatcher::handle_stream` / `dispatch_requested` gains a branch on +`op_type`: + +- `Sub` → `registry.invoke_streaming()` → for each `ResponseEnvelope` + in the stream, write `EventEnvelope` to the wire → write `call.completed` on + stream end. +- `Query` / `Mutation` → `registry.invoke()` → write one `EventEnvelope` + (existing path, unchanged). +- `Pub` → `registry.invoke_sink()` (alkcall ADR-046) → the producer's handler + sinks the incoming `call.published` stream; events flow + producer→consumer via `call.published`. + +The streaming branch sets `deadline: None` for subscriptions (unbounded — +already specced in the alkcall crate's `call-protocol.md` Timeouts) and wires +abort cascade (alkcall ADR-020): if `call.aborted` arrives for a streaming +request, the stream is dropped (Rust `Drop` releases the handler's resources). + +### 7. `GatewayDispatch::invoke_streaming()` (alkhttp) + +The shared dispatch spine gains a streaming variant: + +```rust +impl GatewayDispatch { + pub async fn invoke_streaming( + &self, + identity: Option, + op: &str, + input: Value, + ) -> BoxStream; +} +``` + +`invoke_streaming()` builds the root `OperationContext` identically to +`invoke()` (same security invariants: `internal: false`, `forwarded_for: +None`, same capabilities, same `scoped_env`), then calls +`registry.invoke_streaming()`. The two gateways (`to_openapi`, `to_mcp`) +diverge only on wire-framing; the security axis is provably identical between +`invoke()` and `invoke_streaming()`. + +The HTTP `/subscribe` handler calls `invoke_streaming()` and pipes the +`BoxStream` to SSE: each `Ok(value)` → SSE `data:` frame, +`Err` → SSE error event + close, stream end → close. This replaces the current +one-event `subscribe_stream_from_envelope` with the real streaming path. + +### 8. `from_call` stream forwarding + +The `from_call` forwarding handler construction branches on `op_type` during +discovery: + +- `Query` / `Mutation` → existing `make_forwarding_handler()` (calls + `CallConnection::call_with_payload()`, returns single `ResponseEnvelope`), + registered as `HandlerKind::Once`. +- `Sub` → new `make_streaming_forwarding_handler()` (calls + `CallConnection::subscribe()`, returns `impl Stream`, maps to `BoxStream`), registered as + `HandlerKind::Stream`. + +A `from_call`-imported `Sub` op forwards the remote stream end-to-end: +the consumer-side `CallConnection::subscribe()` (already working) feeds a +`StreamingHandler` that produces the stream. No truncation, no first-value +fallback. + +### 9. `from_openapi` SSE forwarding + +The `from_openapi` forwarding handler construction branches on `op_type` +(determined by `detectOperationType` — `text/event-stream` response → +`Sub`): + +- `Query` / `Mutation` → existing forwarding handler (single HTTP request → + single `ResponseEnvelope`), `HandlerKind::Once`. +- `Sub` → streaming forwarding handler (HTTP request → SSE response + stream → parse SSE chunks → `BoxStream`), `HandlerKind:: + Stream`. + +The SSE parsing reuses the TS `parseSSEFrames` pattern: each SSE `data:` frame +becomes a `ResponseEnvelope::ok()`, SSE stream end becomes stream end (→ +`call.completed`). + +## Consequences + +**Positive:** + +- `Sub` operations work end-to-end: producer-side handler → + producer-side dispatch → wire → HTTP `/subscribe` SSE → `from_call` + forwarding → `from_openapi` SSE forwarding. No truncation, no broken paths. +- The `Handler` / `StreamingHandler` split mirrors the TS prior art exactly, + making the Rust port faithful to its source. +- `HandlerKind` makes the "one or the other, matching `op_type`" invariant + type-level (a `Once` variant for `Query`/`Mutation`, a `Stream` variant for + `Sub`) rather than a runtime check on two `Option`s. +- Existing handlers (echo, discovery, from_openapi Query/Mutation, from_mcp, + from_call Query/Mutation) are unchanged — they return a single + `ResponseEnvelope` and register as `HandlerKind::Once`. The streaming path + is additive to the existing handler surface. +- `OperationEnv` composition stays request/response, preserving the + composition model's simplicity. Stream composition is a handler-level + concern, cleanly separated. +- The new `INVALID_OPERATION_TYPE` protocol code catches dispatch-path misuse + (calling `invoke()` on a `Sub`) at the protocol level instead of + silently producing wrong behavior. + +**Negative:** + +- `HandlerRegistration.handler` changes type from `Handler` to `HandlerKind`. + Existing code constructing `HandlerRegistration` bundles must wrap in + `HandlerKind::Once(...)`. This is a mechanical change across handler + construction sites (the builder's `.with_local()` / `.with_leaf()` / + `.with()` methods absorb the wrapping internally, so most assembly-layer + code is unaffected; direct `HandlerRegistration::new()` calls need the + wrap). +- A new protocol-level error code (`INVALID_OPERATION_TYPE`) is a wire-format + addition. Existing clients that treat unknown codes as `INTERNAL` with + `retryable: false` (the existing rule) handle it correctly — they just + don't distinguish it from `INTERNAL` until updated. The code is distinct + from all existing codes and from operation-level domain codes (no + `HTTP_` prefix, no collision with the five existing protocol codes). +- The `Dispatcher::handle_stream` streaming branch adds a stream-to-wire + pump (read stream → write frames → write `call.completed`). This is new + code in the hot dispatch path, but it is a straightforward `while let + Some(envelope) = stream.next().await` loop, not a complex abstraction. + +## Door type + +**One-way.** The `Handler` / `StreamingHandler` / `HandlerKind` API surface +is what handlers are written against across crates (alkcall, +alkhttp, downstream consumers). Changing it after handlers exist is a +rewrite. The `INVALID_OPERATION_TYPE` wire code is also one-way — once +emitted, clients may handle it, and removing it would break those handlers. + +The `HandlerKind` enum shape (`Once(Handler) | Stream(StreamingHandler)`, +now with the alkcall ADR-046 `Sink(SinkHandler)` third variant) is +the one-way commitment: one handler variant per dispatch model, validated +against `op_type`. The concrete `BoxStream` library choice +(`futures::stream::BoxStream` vs a custom type) is a two-way-door +implementation detail within the one-way decision. + +## References + +- alkcall ADR-015: Call Protocol Stream Model (alknet ADR-012; defines + `subscribe` as a top-level protocol operation; the streaming path this ADR + implements) +- [ADR-017](017-call-protocol-client-and-adapter-contract.md): Call protocol + client and adapter contract (the adapter contract + this ADR extends with the `StreamingHandler` variant; the decision record + is alkcall ADR-022) +- [ADR-022](022-handler-registration-provenance-and-composition-authority.md): + Handler Registration, Provenance, and Composition Authority + (`HandlerRegistration` gains `HandlerKind`; the decision record is alkcall + ADR-018) +- [ADR-015](015-privilege-model-and-authority-context.md): Privilege Model + and Authority Context (visibility/ACL checks run + identically in `invoke_streaming()` as in `invoke()`; the decision record + is alkcall ADR-017) +- alkcall ADR-020: Abort Cascade for Nested Calls (alknet ADR-016; stream + drop on abort; cascade through streaming handlers) +- [ADR-023](023-operation-error-schemas.md): Operation Error Schemas + (`INVALID_OPERATION_TYPE` is a + protocol-level code, distinct from operation-level domain codes; the + decision record is alkcall ADR-016) +- alkcall ADR-024: Peer-Graph Routing Model (alknet ADR-029; `from_call` + forwarding handlers gain the streaming variant) +- alkcall ADR-046: `Pub` operation type and `HandlerKind::Sink` — the + producer→consumer streaming primitive; the third `HandlerKind` variant +- alknet ADR-009: One-Way Door Decision Framework (now alkcall ADR-032; the + `Handler` / `StreamingHandler` split is a one-way door — handler API + surface) +- `@alkdev/operations/src/types.ts:62-78` — TS prior art + (`OperationHandler` / `SubscriptionHandler` split) +- `@alkdev/operations/src/registry.ts:65-75` — TS prior art + (`validateSubscriptionHandler` — runtime validation against `op_type`) +- `@alkdev/operations/src/call.ts:341-349` — TS prior art (`buildCallHandler` + branches on `op_type`: `SUBSCRIPTION` → iterate + complete; else → execute) +- `@alkdev/pubsub/src/operators.ts` — stream operators prior art (filter, + map, batch, dedupe, window, chain, join — handler-level stream + composition, distinct from `OperationEnv` request/response composition) +- Spec documents amended (alknet mono-repo spec tree; the alkhttp equivalents + live in this crate's `docs/architecture/`): `operation-registry.md`, + `call-protocol.md`, `http-server.md`, `http-adapters.md`, `http-mcp.md`, + `client-and-adapters.md` + +## Port notes + +- Renames: "alknet-http" → alkhttp; "alknet-core"/"alknet-call" → alkcall. +- **`OperationType::Subscription` → `Sub`** throughout (alkcall rename; + `OperationSpec.op_type` value and TS-mapping references). The TS prior-art + identifiers (`SUBSCRIPTION`, `SubscriptionHandler`) are left verbatim — + they are the TypeScript package's names, not the Rust enum's. +- **Producer/consumer terminology**: "the **server side** does not" → "the + **producer side** did not (in the original design)"; "The **client side** + works — a client can subscribe" → "The **consumer side** works — a + consumer can subscribe"; "server-side handler → server-side dispatch" → + "producer-side handler → producer-side dispatch"; "the client-side + `CallConnection::subscribe()`" → "the consumer-side"; §4 "client-side + programming error" → "caller-side programming error"; §6 heading + "Server-side dispatch branches on `op_type`" → "Dispatch branches on + `op_type`". HTTP/TS inherent directionality untouched (`buildCallHandler` + is TS prior-art code; `client.ts` citations keep their names). +- **`Pub`/`Sink` note (alkcall ADR-046)**: the original defined two + `HandlerKind` variants (`Once`, `Stream`) — annotated. The §2 code block + now shows the three-variant enum with a block-quote annotation marking the + `Sink` variant as the alkcall ADR-046 addition (decision history + retained); §2's validation sentence and §6's dispatch branch table add the + `Pub` → `invoke_sink()` row with the alkcall ADR-046 citation; the Door + type section's one-way enum-shape sentence updated to acknowledge the + third variant. `from_openapi` produces no `Pub` ops in v1 (SSE detection + yields `Sub`), so §7–§9 need no change. The original two-variant decision + text is otherwise retained verbatim as decision history. +- §7 title: "(alknet-http)" → "(alkhttp)". The `/subscribe` endpoint and + `GatewayDispatch::invoke_streaming()` are this crate's gateway surface + (ADR-042/ADR-047); the Context's first downstream bullet names + `GatewayDispatch::invoke()` → `subscribe_handler`, which is the alkhttp + gateway dispatch spine. +- Cross-reference remappings (verified alknet→alkcall ADR mapping): alknet + ADR-012 (stream model) → alkcall ADR-015; alknet ADR-016 (abort cascade) → + alkcall ADR-020; alknet ADR-022 (handler registration) → alkcall ADR-018; + alknet ADR-017 (adapter contract) → alkcall ADR-022; alknet ADR-015 + (privilege model) → alkcall ADR-017; alknet ADR-029 (peer-graph routing) → + alkcall ADR-024; alknet ADR-009 (one-way door framework) → alkcall ADR-032. + ADR-015/017/022/023 are ported to this crate under the same numbers and + linked; their alkcall record numbers are noted alongside. +- The amended-spec list at the end of References is annotated: those spec + documents live in the alknet mono-repo spec tree (and the alkcall crate's + docs for the call-side ones); the alkhttp equivalents are this crate's + `docs/architecture/http-server.md` and `http-adapters.md`. +- No decision content changed — the `StreamingHandler` type, the + `HandlerKind` enum, `invoke_streaming()`, the `INVALID_OPERATION_TYPE` code, + the `OperationEnv` request/response-only boundary, the dispatch branch, and + the gateway/from_call/from_openapi streaming paths are verbatim from the + alknet ADR modulo the corrections logged above. \ No newline at end of file diff --git a/docs/architecture/decisions/051-yaml-input-for-from-openapi.md b/docs/architecture/decisions/051-yaml-input-for-from-openapi.md new file mode 100644 index 0000000..eb8925f --- /dev/null +++ b/docs/architecture/decisions/051-yaml-input-for-from-openapi.md @@ -0,0 +1,241 @@ +# ADR-051: YAML Input Format for from_openapi + +*Ported from alknet ADR-051 (YAML Input Format for from_openapi); re-targeted to alkhttp.* + +## Status + +Accepted + +## Context + +`from_openapi` imports external HTTP APIs as call-protocol operations by +parsing an OpenAPI document. The `http-adapters.md` spec (now in this crate's +`docs/architecture/`) already states the +one-way constraint as "`from_openapi` accepts a standard OpenAPI 3.x +JSON/YAML doc" — YAML was always part of the intended input contract. The +implementation, however, only ever delivered the JSON half: +`OpenAPISpec::from_json(&str)`. This was fine for providers that publish JSON +OpenAPI schemas (e.g., runpod's `openapi.json`) but blocks providers that +publish YAML schemas (e.g., vast.ai's `openapi.yaml`). A coming consumer crate +needs to import vast.ai's operations, which surfaces the gap. + +This is gap-filling against an existing constraint, not a new architectural +direction. None of the architecture invariants are touched: `OpenAPISpec` +stays `serde_json::Value`-based, the forwarding handler is unchanged, the +no-env-vars credential injection is unchanged, error fidelity is unchanged, +`to_openapi` output stays JSON. The change is a new parse path into the +same internal type. + +Two decisions need recording: the parse strategy (JSON-first, not +parse-everything-as-YAML) and the dependency choice (the maintained +`yaml_serde` fork, not the deprecated `serde_yaml`). + +## Decision + +### 1. `from_openapi` accepts YAML via a `from_yaml` constructor and a format-detecting `from_str` + +`OpenAPISpec` gains two constructors alongside the existing `from_json`: + +```rust +impl OpenAPISpec { + pub fn from_json(doc: &str) -> Result; // existing + pub fn from_yaml(doc: &str) -> Result; // new — parses YAML + pub fn from_str(doc: &str) -> Result; // new — detects format + pub fn from_value(raw: Value) -> Result; // existing, unchanged +} +``` + +`from_str` is the convenience for callers that have a raw doc string of +unknown format (e.g., fetched from a URL with no Content-Type hint). The +detection rule is **JSON-first, YAML-fallback** (see §2 for why the order +matters): attempt `serde_json::from_str`; if it parses, use the result; if +it fails, attempt YAML. `from_json` and `from_yaml` remain for callers that +know the format and want a precise error on mismatch. + +This is an additive API surface change (two-way door — constructors can be +renamed/added; nothing downstream breaks). The constructors produce the same +`OpenAPISpec`; the rest of the adapter is format-agnostic. + +### 2. Format detection is JSON-first, YAML-fallback — a defensive default, not a style preference + +> **Amendment (2026-07-06):** The original §2 cited YAML 1.1 +> boolean-coercion (`yes`/`no`/`on`/`off` → booleans) as a *present +> hazard* with the maintained Rust YAML crates, framing JSON-first as a +> correctness guard against silent string→boolean mutation. A probe +> during implementation verified this is factually wrong for the chosen +> dependency: `yaml_serde` 0.10.x (and the deprecated `serde_yaml` 0.9) +> implement the **YAML 1.2 core schema**, where only `true`/`false` (and +> case variants) are booleans — the bare tokens `yes`/`no`/`on`/`off`/ +> `y`/`n` are plain strings. The coercion hazard the original rationale +> cited does not exist with this dependency version. The JSON-first rule +> is **retained** (Accepted ADR) — the rationale is reframed below as a +> defensive default, not a guard against a present hazard. The decision +> did not change; the rationale did. + +JSON's grammar is a strict subset of YAML (under YAML 1.2) and never +exhibits any YAML-specific type interpretation. Running a JSON document +through a YAML parser is *currently* safe with `yaml_serde` 0.10.x — a +JSON doc like `{"active": "yes"}` parses through the YAML path with +`"yes"` intact as a string (YAML 1.2 core schema, verified by the +`from_yaml_preserves_bare_yes_as_string_yaml_1_2_behavior` test). +JSON-first detection is therefore not guarding against a present hazard +with this dependency; it is a **defensive default that locks in the +contract against a future YAML-parser swap**. If `yaml_serde` is ever +swapped for a YAML 1.1 crate (where `yes`/`no`/`on`/`off` coerce to +booleans), or if a future `yaml_serde` version tightens its core schema +in a way that introduces type interpretation JSON doesn't have, the +JSON-first rule ensures JSON input cannot be silently mutated by the +YAML path. The contract is durable; the dependency is a two-way door +(§3). + +The rule is cheap: `from_str` tries `serde_json::from_str` first (strict +grammar, no YAML-specific interpretation), and only on JSON parse failure +falls back to the YAML parser. A YAML-only document (with `openapi: 3.0.0` +at the top, no JSON braces) fails JSON parse immediately and goes to the +YAML path. The cost is one wasted parse attempt for YAML docs, paid once +at adapter-import time (not per forwarded call — see Consequences). + +`from_yaml` (the explicit constructor) does not try JSON first — the caller +has declared the format. This is correct: a caller that explicitly says +"this is YAML" wants the YAML parse, including whatever type +interpretation the YAML parser applies. If the caller is wrong (passes +JSON to `from_yaml`), the YAML parser handles it — JSON is a syntactic +subset of YAML, so it parses, with whatever interpretation the YAML +parser's schema applies (currently none for `yes`/`no` under YAML 1.2; a +future YAML 1.1 swap would coerce). The caller opted in by naming the +format; `from_str` exists for the unsure caller. + +### 3. The YAML dependency is `yaml_serde` (the official YAML org fork of `serde_yaml`), not the deprecated `serde_yaml` + +The original `serde_yaml` crate (dtolnay) is no longer maintained. The +official [YAML organization](https://github.com/yaml) maintains a +continuation published as `yaml_serde` (crate name `yaml_serde`, v0.10), a +drop-in fork with full API compatibility. The migration path is either +`serde_yaml = { package = "yaml_serde", version = "0.10" }` (keeps +`use serde_yaml::` imports) or direct `yaml_serde = "0.10"` with updated +imports. alkhttp uses the direct form (`yaml_serde = "0.10"`, +`use yaml_serde::`). + +The dependency is a two-way door: `yaml_serde` can be swapped for another +maintained YAML-serde fork (or a future replacement) by changing the +Cargo line and the imports. The one-way constraint is that alkhttp +owns its YAML parse and produces `serde_json::Value` (the shared internal +type) — which dependency does the parse is an implementation detail. +`yaml_serde` is chosen because it is the maintained continuation under +the official YAML umbrella, not because its API is irreversibly +load-bearing. + +The dependency is **not feature-gated**. YAML OpenAPI schemas are a +first-class input format (vast.ai publishes one), not an edge case. Gating +it behind a feature would mean a deployment that imports vast.ai must +remember to enable the feature — the kind of friction the no-surprises +default-features model avoids. The dependency is small (a pure-Rust YAML +parser, no native code), consistent with the existing default-features +philosophy of the crate. + +### 4. Scope boundary: `to_openapi` output is not affected + +`to_openapi` generates the published gateway doc, served at `GET +/openapi.json`. It stays JSON. This ADR fills a gap on the *consume* side +(importing external YAML schemas); the *publish* side serves our own +gateway contract and JSON is the standard exchange format for OpenAPI +tooling (code generators, validators, `fetch`-based clients all consume +JSON). A `GET /openapi.yaml` additive output is not part of this decision: +it is a separate scope (publish-side format, not consume-side), would be a +separate ADR if a concrete consumer requires YAML output, and is +additive (a new endpoint, no breaking change to the JSON path). The +`OpenAPISpec` type is shared, but the output serialization is JSON-only. + +## Consequences + +**Positive:** +- `from_openapi` consumes both JSON and YAML OpenAPI schemas — the + intended contract (spec line: "JSON/YAML doc") is finally delivered. vast.ai + and any other YAML-publishing provider can be imported. +- Format detection (`from_str`) makes fetch-and-import ergonomic: a caller + that fetched a schema from a URL with no reliable Content-Type doesn't + have to sniff the format itself. +- JSON-first detection is a defensive default that locks in the + contract against a future YAML-parser swap. With `yaml_serde` 0.10.x + (YAML 1.2 core schema) the coercion hazard the original rationale + cited is not present; JSON-first nonetheless ensures JSON input is + never exposed to YAML-specific type interpretation, present or + future. The rule is cheap (one wasted parse for YAML docs, paid once + at import time) and the contract is durable. +- The maintained `yaml_serde` fork keeps the dependency off the archived + `serde_yaml`; the swap is documented so a future maintainer doesn't + re-derive why the crate name doesn't match the obvious name. + +**Negative:** +- A new pure-Rust dependency (`yaml_serde`) in alkhttp. Small, but + non-zero. The trade is first-class YAML support without a feature gate — + accepted because YAML OpenAPI is a real input format, not an edge case. +- `from_str`'s JSON-first detection does one wasted parse attempt for YAML + docs (the JSON parse fails, then the YAML parse runs). The cost is + trivial — `from_openapi` runs once at adapter-import time (not per + forwarded call), so the double-parse happens once per imported service, + not per request. Callers that know the format use `from_json`/`from_yaml` + directly and pay no double-parse. The defensive benefit (JSON input never + reaches the YAML parser, immune to any YAML-specific interpretation + present or future) is worth the one-time cost. + +## Assumptions + +1. **The `OpenAPISpec` internal type stays `serde_json::Value`-based.** YAML + parses to `serde_json::Value` via `yaml_serde`, then feeds the existing + `from_value` path. No second internal representation. If a future + switch to `openapiv3::OpenApi` happens (the two-way-door the spec already + notes), both JSON and YAML constructors adapt in lockstep — the + constructor is the adapter between wire format and internal type. + +2. **`yaml_serde` 0.10.x implements the YAML 1.2 core schema.** Verified + by a probe during implementation: bare `yes`/`no`/`on`/`off`/`y`/`n` + are plain strings, not booleans (codified by the + `from_yaml_preserves_bare_yes_as_string_yaml_1_2_behavior` test). The + original §2 rationale cited YAML 1.1 coercion as a present hazard; it + is not, with this dependency version. JSON-first detection is retained + as a defensive default (§2 as amended): a future swap to a YAML 1.1 + crate, or a future `yaml_serde` schema tightening, cannot silently + regress JSON input because JSON never reaches the YAML path under + `from_str`. If the dependency swaps to a YAML 1.1 crate, the defensive + default becomes a load-bearing correctness guard — the rule is the + same either way, which is why it is stated as a contract rather than + as a workaround for a specific crate version. + +## References + +- [ADR-017](017-call-protocol-client-and-adapter-contract.md) — + `from_openapi` is an `OperationAdapter`; published `to_*` specs are + compatibility contracts (the publish side stays JSON; the decision record + is alkcall ADR-022) +- [ADR-023](023-operation-error-schemas.md) — error fidelity is unaffected + (error schemas come from the parsed `OpenAPISpec`, format-independent; the + decision record is alkcall ADR-016) +- [ADR-039](039-http-server-and-client-host-colocated.md) — alkhttp + owns both HTTP directions and their dependencies +- [http-adapters.md](../http-adapters.md) — the spec that + states the "JSON/YAML doc" constraint, the `OpenAPISpec` type, and the + Constraints/Design Decisions entries this ADR backs (see the + "Input formats" doc-comment, the Constraints §"`from_openapi` accepts + JSON and YAML", and the Design Decisions table row; now in + `docs/architecture/`) +- `yaml_serde` crate (https://github.com/yaml/yaml-serde) — the maintained + official-YAML-org fork of the deprecated `serde_yaml` + +## Port notes + +- Renames: "alknet-http" → alkhttp throughout (§3's "the dependency is not + feature-gated" paragraphs, Consequences, §3's "alknet-http uses the direct + form" → "alkhttp uses the direct form"). +- Link path fixes: `../crates/http/http-adapters.md` → `../http-adapters.md` + (the spec now lives in this crate's `docs/architecture/`). Cross-ADR links + (ADR-017, ADR-023, ADR-039) point at `decisions/NNN-.md` with the + alknet slugs; ADR-017 and ADR-023 are ported here under the same numbers + (their decision records live in alkcall as ADR-022 and ADR-016 + respectively — noted inline). +- No producer/consumer or `Sub`/`Pub` terminology appears in the original — + nothing to retarget on those axes. +- No decision content changed — the constructor set, the JSON-first + YAML-fallback rule, the `yaml_serde` dependency choice and its + non-feature-gating, the 2026-07-06 rationale amendment, and the + `to_openapi`-stays-JSON scope boundary are verbatim from the alknet ADR. \ No newline at end of file diff --git a/docs/architecture/decisions/066-from-jsonschema-as-http-adapter.md b/docs/architecture/decisions/066-from-jsonschema-as-http-adapter.md new file mode 100644 index 0000000..8a05c29 --- /dev/null +++ b/docs/architecture/decisions/066-from-jsonschema-as-http-adapter.md @@ -0,0 +1,240 @@ +# ADR-066: `from_jsonschema` as an HTTP-Backed Single-Endpoint Adapter in alkhttp + +*Ported from alknet ADR-066 (`from_jsonschema` as an HTTP-Backed Single-Endpoint Adapter in alknet-http); re-targeted to alkhttp.* + +## Status + +Accepted (supersedes the `from_jsonschema` clause of ADR-017 §5 and the +`FromJsonSchema` provenance row of ADR-022 — both described a schema-only, +no-handler adapter in the call crate) + +## Context + +`from_jsonschema` was originally specified (alknet ADR-017 §5) as a +schema-only +adapter living in the call crate (`alknet-call`, now alkcall): it produced +`HandlerRegistration` bundles +with a `NOT_FOUND`-returning placeholder handler and `FromJsonSchema` +provenance. The stated use case was validation, discovery, and +composition-graph construction without a runtime — type-checking a +composition plan without executing it, building a UI of available +operations without standing up the transports. + +This is broken. An operation in the `OperationRegistry` needs a real +handler. A placeholder that returns `NOT_FOUND` does not work with how +the registry is supposed to function: an `Internal` op registered with +a dead handler is a trap, not a feature. The "schema-only, no handler" +concept conflated two things — schema *validation* (a compile-time / +planning activity that doesn't need a registry entry at all) and +operation *registration* (which always needs a handler). Validation +against a JSON Schema does not require a `HandlerRegistration`; it +requires the schema and a validator. Registering an operation requires +a handler. The old `from_jsonschema` tried to do the former by abusing +the latter, and produced something that works for neither. + +The misplacement was compounded by a location error: the adapter lived +in the call crate (which is supposed to stay lean — no HTTP client), but +a `from_jsonschema` that is actually useful for calling non-standard +endpoints needs reqwest, exactly like `from_openapi` and `from_mcp`. +The adapter location map in ADR-017 / the call crate's +`client-and-adapters.md` already +establishes that HTTP-backed adapters live in the HTTP crate; the old +`from_jsonschema` violated its own stated principle by living in +the call crate. The move was recorded as alknet ADR-066 (moved from +`alknet-call` to `alknet-http`); with the extraction, that makes it this +crate — alkhttp. The `FromJsonSchema` provenance variant itself stays in +the call crate (alkcall ADR-027 records the provenance-side decision: +`FromJsonSchema` is a handler-bearing leaf in alkcall's +`OperationProvenance` enum). + +A concrete use case now forces the decision: composing a non-standard, +non-OpenAPI, basic REST endpoint that does not have a full OpenAPI +document. The endpoint has a method, a URL, an input/output JSON Schema, +and an auth scheme — but no `paths` object, no `operationId`, no +`components`. `from_openapi` requires an OpenAPI document; this endpoint +doesn't have one. The gap is: register a single HTTP endpoint as a +call-protocol operation, one at a time, with the caller supplying the +schema directly. + +## Decision + +`from_jsonschema` becomes an HTTP-backed single-endpoint adapter in +alkhttp, functionally similar to `from_openapi` but registering +one endpoint at a time instead of parsing a full OpenAPI document: + +1. **The adapter implementation lives in alkhttp** + (`src/adapters/from_jsonschema.rs`). The + forwarding handler uses the same reqwest-backed `SharedHttpClient` + and the same no-env-vars credential injection as `from_openapi`. The + adapter implements `OperationAdapter` (the trait from alkcall, + ADR-017 §5 — unchanged). + +2. **Give it a real forwarding handler.** A `from_jsonschema`-imported + operation is a leaf with a reqwest forwarding handler, identical in + shape to a `from_openapi`-imported operation — it builds an HTTP + request from the input (path/query/body split per a path template), + injects credentials from `context.capabilities`, sends via the shared + HTTP client, and parses the response (JSON, text, or binary — same + content-type branching as `from_openapi`). For a `Sub` + op type with `text/event-stream` response, it registers a + `StreamingHandler` (ADR-049), same as `from_openapi`. + +3. **Single-endpoint registration.** The caller supplies: + - An `OperationSpec` (name, op type, input/output JSON Schema, + `error_schemas`, `access_control`, `visibility`). + - An `HttpServiceConfig` (base URL, auth scheme, default headers — + the same config type `from_openapi` uses). + - A path template + HTTP method (the one endpoint). + + The adapter builds one `HandlerRegistration` with `FromJsonSchema` + provenance and a real forwarding handler. The caller registers it in + the `OperationRegistry`. This is the "one endpoint at a time" shape: + no `paths` object to iterate, no `operationId` to normalize. + +4. **`FromJsonSchema` provenance stays in alkcall** (in the + `OperationProvenance` enum, `registration.rs`). The provenance type + lives where the registry types live; only the adapter implementation + moved. `FromJsonSchema` is now a leaf provenance — it has a handler + (a reqwest forwarding handler), same trust model as `FromOpenAPI` + (HTTP endpoint trusted; handler is a forwarding stub). + +5. **Remove the "schema-only, no handler" concept.** The placeholder + handler and the "schema-only ops are `Internal`, so dispatch should + never reach them" rationale are removed. An op registered with + `FromJsonSchema` provenance is a real, callable, HTTP-forwarding + operation — `Internal` by default (adapter-registered ops are + composition material, ADR-015), but it actually forwards if invoked. + + The schema-validation-without-a-handler use case (type-checking a + composition plan, building a UI) does not require a + `HandlerRegistration` at all. That use case is served by consuming + the `OperationSpec` directly (the spec already carries the input/ + output JSON Schemas); no adapter, no registry entry, no handler is + needed. If a future use case requires registering a schema-only op + for discovery purposes, that is a separate feature and would warrant + its own ADR — it is not what `from_jsonschema` is. + +### Relationship to `from_openapi` + +| | `from_openapi` | `from_jsonschema` | +|---|---|---| +| Input | A full OpenAPI 3.x document (JSON or YAML) | A single endpoint: `OperationSpec` + `HttpServiceConfig` + path template + method | +| Granularity | One `HandlerRegistration` per `(path, method)` in the doc | One `HandlerRegistration` per call | +| Schema source | Parsed from the OpenAPI doc (parameters, request body, responses) | Supplied directly by the caller | +| Handler | reqwest forwarding handler (shared HTTP client) | Same reqwest forwarding handler | +| Provenance | `FromOpenAPI` | `FromJsonSchema` | +| Location | alkhttp | alkhttp | +| Use case | Standard OpenAPI APIs (GitHub, OpenAI, Anthropic) | Non-standard, non-OpenAPI, or basic REST endpoints without a full spec | + +The two adapters share the forwarding-handler implementation, the +credential injection path, the error-fidelity rule (`HTTP_` +prefix, [ADR-023](023-operation-error-schemas.md)), and the no-env-vars +invariant ([ADR-014](014-secret-material-flow-and-capability-injection.md)). +The +difference is purely the input shape: a full document vs. a single +endpoint. + +## Consequences + +**Positive**: +- `from_jsonschema` actually works — it has a real handler, not a + placeholder. A concrete use case (non-standard REST endpoints) is + served. +- The adapter location is consistent: all HTTP-backed adapters + (`from_openapi`, `from_mcp`, `from_jsonschema`) live in the HTTP crate + (alkhttp), + where reqwest is. The call crate (alkcall) stays lean. +- The "schema-only, no handler" trap is removed. An op in the registry + is always callable. +- `FromJsonSchema` provenance becomes a real leaf, consistent with + `FromOpenAPI`/`FromMCP`/`FromCall`. + +**Negative**: +- The schema-validation-without-a-handler use case (the original stated + purpose) is no longer served by `from_jsonschema`. That use case is + served by consuming `OperationSpec` directly, but any code that relied + on the placeholder handler returning `NOT_FOUND` breaks. The only + existing consumer was the call crate's own tests; no downstream consumer + depended on this — the placeholder was a trap, not a contract. +- The call crate loses a public export (`from_jsonschema`, `FromJsonSchema` + the adapter struct). The `FromJsonSchema` provenance variant stays; + the adapter struct moves. Downstream consumers that referenced the + adapter (none currently) would need to use alkhttp's re-export. + +**Neutral**: +- `FromJsonSchema` provenance is now a leaf (handler-bearing), not a + "no handler" provenance. The ADR-022 table row updates: it can compose? + No. Has composition authority? No. Default visibility? Internal. Trust + model? HTTP endpoint trusted; handler is a forwarding stub. This + aligns with the other leaves. ADR-017 §5 and ADR-022's provenance + table/enum-doc are amended (2026-07-09) to point here — the + supersession is recorded in the superseded ADRs, not only in this one. + +## References + +- Supersedes the `from_jsonschema` clause of + [ADR-017](017-call-protocol-client-and-adapter-contract.md) §5 + ("`FromJsonSchema` — imports from a JSON Schema definition (schema-only, + no handler)") and the operational spec in the call crate's + `client-and-adapters.md` §"from_jsonschema" (alknet mono-repo: + `docs/architecture/crates/call/client-and-adapters.md`). +- Supersedes the `FromJsonSchema` row of + [ADR-022](022-handler-registration-provenance-and-composition-authority.md) + (the "no handler — schema only" framing). +- Aligns with the adapter location principle in + [ADR-017](017-call-protocol-client-and-adapter-contract.md) §5 and the + call crate's `client-and-adapters.md` §"Adapter Location Map": HTTP-backed + adapters live in the HTTP crate (alkhttp). +- Reuses the forwarding handler, credential injection, error fidelity + (`HTTP_` prefix, [ADR-023](023-operation-error-schemas.md)), + streaming shape ([ADR-049](049-streaming-handler-for-subscriptions.md)), + and no-env-vars invariant ([ADR-014](014-secret-material-flow-and-capability-injection.md)) + established by `from_openapi`. +- Reuses `HttpServiceConfig` and `SharedHttpClient` from + `from_openapi` (in alkhttp). +- alkcall ADR-027 — the decision record in the call crate (`from_jsonschema` + as an HTTP-backed adapter; `FromJsonSchema` provenance is a + handler-bearing leaf in alkcall's `OperationProvenance`). + +## Port notes + +- **History accuracy (kept per the porting instruction):** the move this + ADR records happened in two hops. Originally the adapter was specified in + the call crate (`alknet-call`) as a schema-only placeholder; alknet + ADR-066 moved it to `alknet-http` (the HTTP crate of the alknet + mono-repo); with the crate extraction, `alknet-http` is now **alkhttp**. + The title, §Decision 1, the location table, and the Consequences now + name alkhttp as the home. The original title said "in alknet-http"; the + ported title says "in alkhttp" — same crate, current name. +- **Provenance location (per alkcall ADR-027):** `FromJsonSchema` provenance + lives in alkcall (the call crate's `OperationProvenance` enum) — the + provenance is a handler-bearing leaf in alkcall; only the adapter + implementation lives in alkhttp. The original already said this; the port + names the current crates and adds the alkcall ADR-027 citation to the + References. +- Renames: "alknet-http" → alkhttp; "alknet-call" → "the call crate + (alkcall)"; the source-file path `crates/alknet-http/src/adapters/from_jsonschema.rs` + → `src/adapters/from_jsonschema.rs` (this crate's layout); "the HTTP + crate" phrasing retained where the original used it generically. +- `Subscription` → `Sub` (alkcall rename; alkcall ADR-046 added + `OperationType::Pub` — producer→consumer streaming via `call.published`, + `HandlerKind::Sink` — which does not affect this adapter: `from_jsonschema` + detects SSE responses as `Sub` and registers `HandlerKind::Stream`, same + as `from_openapi`). +- Cross-reference remappings (verified alknet→alkcall ADR mapping): alknet + ADR-017 (adapter contract) → alkcall ADR-022; alknet ADR-022 (handler + registration) → alkcall ADR-018; alknet ADR-023 (error schemas) → alkcall + ADR-016; alknet ADR-049 (streaming handler) → alkcall ADR-021; alknet + ADR-014 (secret material flow) → alkcall ADR-010. ADR-014/017/022/023/049 + are ported to this crate under the same numbers and linked. +- The `client-and-adapters.md` references became textual "the call crate's + `client-and-adapters.md`" references with the alknet mono-repo path noted + (the document lives in alkcall's docs/architecture/). +- Status line: the superseded clauses (ADR-017 §5, ADR-022's + `FromJsonSchema` row) are cited as this crate's ported ADRs — the + supersession text is unchanged from the alknet original. +- No decision content changed — the real-forwarding-handler requirement, the + single-endpoint registration shape, the provenance location, the removal of + the schema-only concept, the comparison table, and the + neutral/positive/negative consequences are verbatim from the alknet ADR + modulo the corrections logged above. \ No newline at end of file diff --git a/docs/architecture/decisions/067-websocket-carries-channels.md b/docs/architecture/decisions/067-websocket-carries-channels.md new file mode 100644 index 0000000..6181394 --- /dev/null +++ b/docs/architecture/decisions/067-websocket-carries-channels.md @@ -0,0 +1,170 @@ +# ADR-067: WebSocket Carries the Channels Protocol + +## Status + +Accepted + +## Context + +The alknet design (ADR-044, ADR-048 there) made the WebSocket path a +bare call-protocol session: one `EventEnvelope` JSON object per binary +WS message, handed directly to the shared `Dispatcher`. That design +predates the channels protocol — the 8-byte chunk multiplexer (alkcall +ADR-034/035) with channel 0 pre-negotiated as `alk/call` (alkcall +ADR-036) — and was specified when browsers needed only the call +protocol. + +alkhttp is extracted onto alkcall, where the call protocol and the +channels protocol are one crate, one connection model, and one wire +story. Two problems remain with the bare-envelope WS design: + +1. **No data channels for browsers.** A browser session over WS could + reach the call protocol but could never open a data channel (TTY, + tunnel, a WASM SSH client's transport). Every downstream protocol + crate that rides channels would need a separate browser path. +2. **Two connection shapes, one stack.** A browser WS session and a + Rust in-line channels session (TCP+TLS) would be structurally + different sessions at the dispatch layer: the browser one a raw + envelope stream, the Rust one a channels connection with a + `ChannelManager`. Hub code that wants to treat browser sessions and + Rust spokes uniformly would branch on the transport. + +## Decision + +**The WebSocket path carries the channels protocol, not a bare +envelope stream.** A WS session is an **in-line channels substrate** +(alkcall ADR-034 §substrate modes): the WS connection's binary message +stream is the transport; the 8-byte chunk header demultiplexes N +logical channels over it; **channel 0 is pre-negotiated as `alk/call`** +(alkcall ADR-036) and carries the native call-protocol session — the +shared `Dispatcher` runs on it unchanged. + +### Upgrade path + +The default WS upgrade path is **`/alk/channels`** (was `/alknet/call` +in the alknet design). The path is an axum route on the `HttpAdapter` +router, subject to the same reserved-path collision rule as any +default-surface route ([ADR-046](046-assembly-layer-custom-http-routes.md)). + +### Framing: the WS message boundary carries chunks, not envelopes + +The alknet design's "one `EventEnvelope` = one binary WS message, no +length prefix" framing is **superseded**. The WS message boundary now +carries channels chunks; the call protocol's envelopes ride inside +channel 0 as length-prefixed JSON (alkcall ADR-014 frame format), the +same as any other in-line channels transport. + +Layering on the wire, for a call frame over WS: + +``` +WS binary message +└── chunk header [channel_id: u32 BE][length: u32 BE] (8 bytes) + └── payload = frame [len: u32 BE][EventEnvelope JSON] (channel 0) +``` + +A data channel's chunks are `chunk header + opaque payload` — the +handler (TTY, tunnel, WASM client) owns its framing inside the payload, +exactly as over TCP+TLS (alkcall ADR-035: no `stream_type`, the +handler owns its sub-stream multiplexing). + +The WS↔byte-stream adaptation (how the message-oriented WS stream +presents as the byte stream the channels demux reads, and how the mux's +byte writes become WS messages) is the implementation's core piece and +its buffering semantics are tracked in OQ-01. + +### Dispatch: channel 0 = the shared `Dispatcher`, unchanged + +On upgrade, the handler: + +1. Resolves the caller's identity from the `Authorization: Bearer` + header via `IdentityProvider::resolve_from_token()` — the same auth + path as any HTTP request ([ADR-004](004-auth-as-shared-core.md)). + No token → `401`. The resolved identity is carried on the session + for observability and `AccessControl`. +2. Wraps the WS stream as a `Connection` (`Connection::from_bidi`, ALPN + `alk/channels`). +3. Runs the channels accept path — alkcall's `ChannelsAdapter` + in-line demux loop — installing channel 0 via the + `install_channel_zero` hook: construct channel 0's `CallConnection` + and run `Dispatcher::run_loop_single_stream` on it, exactly as the + TCP+TLS in-line substrate does. +4. Data channels (1..N) are routed to whatever the deployment + registered as openable ALPNs — for a hub, the same openable ALPNs a + Rust spoke can reach. The browser opens them via the per-ALPN open + ops on channel 0 (alkcall ADR-047), the same mechanism as any + consumer. + +Everything ADR-048 says about dispatch — `call.requested` → +`Dispatcher::dispatch_requested` with `AccessControl::check` gating, +`call.responded`/`call.completed`/`call.aborted` correlated by `id` via +the pending map, text WS messages rejected with a protocol-level close +— applies to channel 0 verbatim. The only framing change is the +envelope's position in the layering (above). + +### Bidirectionality, overlay, browsers-are-not-peers: unchanged + +- **Both sides can initiate calls** on channel 0 (alkcall ADR-015's + stream-agnostic correlation). The browser calls hub ops; the hub can + call browser-registered ops over the same session. +- **Browser-registered ops land in the connection-local Layer 2 + overlay** (alkcall ADR-019) and die when the WS connection drops. +- **Browsers are not peers** ([ADR-034](034-outgoing-only-x509-and-three-peer-roles.md) + §4): bearer token, no `PeerId`, not in the peer graph. The connection + handle, not a `PeerRef`, is how the hub reaches browser ops. + +### What this gives the browser + +A browser session is now structurally the same session as a Rust +in-line channels session: + +- Call protocol operations (channel 0) — both directions. +- Data channels for anything the deployment registers as openable — + a WASM SSH client, a TTY, a tunnel — with no browser-specific + protocol work. +- The same discovery ops (`services/list`, `services/schema`) as any + consumer. + +## Consequences + +**Positive:** + +- One session shape across browser and Rust in-line transports; hub + code is transport-blind. +- Browsers get data channels for free, riding alkcall's channels + machinery rather than a bespoke browser protocol. +- The `from_wss` consumer adapter ([ADR-070](070-from-wss-consumer-adapter.md)) + becomes the mirror image of the server path: the same WS byte-stream + adaptation, then alkcall's `ChannelClient`. +- The WS framing delta from alknet is contained to alkhttp's WS adapter; + the Dispatcher, registry, and channels layers are untouched (they are + alkcall's). + +**Negative:** + +- Browsers must speak the channels framing (8-byte header, channel 0 + open ops) rather than bare JSON envelopes. The JS client for this + lives outside alkhttp; the framing is small and the alkcall BAST + document (`chunk-header.bast.json`) is the cross-language contract. +- The WS↔byte-stream adapter is new, unproven code with backpressure + and partial-write hazards (OQ-01). +- Existing alknet-era WS clients (bare envelope per message) break. No + such client ships outside the mono-repo; the break is intentional and + one-way. + +## References + +- [websocket.md](../websocket.md) — the full WS session spec +- [ADR-048](048-websocket-native-session-not-gateway.md) — the native + session (not gateway shape) decision this ADR amends (channel 0 + framing; upgrade path) +- [ADR-044](044-defer-webtransport-browsers-use-websocket.md) — WS as + the browser bidirectional path (stands); deferral mechanics + superseded by [ADR-069](069-webtransport-out-of-scope.md) +- [ADR-070](070-from-wss-consumer-adapter.md) — the consumer-side + mirror of this decision +- alkcall ADR-014 (EventEnvelope framing), ADR-015 (stream model), + ADR-034/035 (channels wire format, pure multiplexing), ADR-036 + (channel 0 pre-negotiated `alk/call`), ADR-039 (ChannelsAdapter), + ADR-043 (ChannelClient), ADR-047 (openable ALPNs are operations) +- alkcall `docs/architecture/channels-wire.md` — the 8-byte chunk + format and wire invariants \ No newline at end of file diff --git a/docs/architecture/decisions/068-gateway-publish-endpoint.md b/docs/architecture/decisions/068-gateway-publish-endpoint.md new file mode 100644 index 0000000..6e832bb --- /dev/null +++ b/docs/architecture/decisions/068-gateway-publish-endpoint.md @@ -0,0 +1,136 @@ +# ADR-068: Gateway `/publish` Endpoint for Pub Operations + +## Status + +Accepted + +## Context + +alkcall added `OperationType::Pub` and `HandlerKind::Sink` +(alkcall ADR-046): producer→consumer streaming, where the *initiator* +streams chunks to the operation via `call.published` events and the +handler consumes them as a `PublishStream`. This is the inverse of +`Sub` (consumer→producer streaming of results) and completes the four +operation types: `Query`, `Mutation`, `Sub`, `Pub`. + +The HTTP gateway ([ADR-042](042-openapi-gateway-pattern.md), +[ADR-047](047-remove-direct-call-http-surface.md)) exposes five +endpoints: `/search`, `/schema`, `/call`, `/batch`, `/subscribe`. The +gateway's dispatch covers `Query`/`Mutation` (`/call`, `/batch`) and +`Sub` (`/subscribe` via the SSE projection, +[ADR-049](049-streaming-handler-for-subscriptions.md)). `Pub` +operations have no HTTP expression: an HTTP client cannot feed a +`Pub` operation's sink. + +Without a surface, `Pub` operations are call-protocol-only (WS channel +0 or a QUIC/`alk/channels` session). That leaves HTTP clients — +curl, axios, a server-side script — unable to produce into `Pub` ops, +which breaks parity: every operation type reachable over the call +protocol should be reachable over HTTP, or the HTTP surface is not a +faithful projection. + +## Decision + +**The gateway gains a sixth endpoint: `POST /publish`.** It invokes a +`Pub` operation; the HTTP request body is the initiator's publish +stream; the operation's final `ResponseEnvelope` is the HTTP response. + +### Request + +- Path: `POST /publish`. +- Body: **newline-delimited JSON (NDJSON)** — each line is one + published chunk, serialized as a JSON value; the stream of lines maps + 1:1 to `call.published` events. Chunk boundaries are the line + boundaries; a line's JSON value is the chunk payload. +- Auth: `Authorization: Bearer ` — same as every gateway + endpoint ([ADR-004](004-auth-as-shared-core.md)). +- The target operation is named the same way as `/call` — the body's + first line (or a `?operation=` query parameter) carries + `{ "operation": "/{service}/{op}", "chunk": {...} }` for the first + chunk, with subsequent lines carrying `chunk` values only; see OQ-02 + for the exact first-line convention before the gateway-spec version + bumps. + +### Dispatch + +1. Resolve identity (Bearer → `resolve_from_token`). +2. Look up the operation; enforce `Visibility::External` (Internal → + `404`, same as `/call`) and `AccessControl::check` (→ `403`). +3. Verify `op_type == Pub` — a non-`Pub` op is + `INVALID_OPERATION_TYPE` → `400`. +4. Dispatch through `invoke_sink()` (alkcall ADR-046): stream each + NDJSON line as one `call.published` chunk into the handler's + `PublishStream`. +5. On end-of-body, deliver the handler's final `ResponseEnvelope`: + - `Ok(output)` → `200` with the output as JSON. + - `Err(call_error)` → mapped status per the standard error mapping + ([ADR-023](023-operation-error-schemas.md), the gateway's + `HTTP_` fidelity rules). + +### Wire-shape note + +On the call protocol, the initiator's chunks are `call.published` +events over channel 0 or a stream; abort is `call.aborted`. Over +HTTP, the abort path is the request being cut short: the client +closing the connection early drops the body stream — the dispatch +cancels the sink (the handler's `PublishStream` sees EOF, matching +write-half close semantics). + +### `to_openapi` projection + +The published gateway doc ([ADR-045](045-to-openapi-gateway-spec-versioning.md)) +describes `/publish` alongside the other five endpoints. The addition +bumps the gateway contract's minor version. The per-caller operation +surface remains discovered via `/search` (`Pub` ops are listed there); +the doc does not preload operations. + +### What does not change + +- `to_mcp` still exposes 4 tools and excludes both `Sub` and `Pub` + ([ADR-041](041-mcp-tool-gateway-pattern.md); alkcall ADR-046) — MCP + tool calls are request/response. +- The WS path needs no `/publish` equivalent: a WS session's channel 0 + carries native `call.published` events. +- `from_openapi`/`from_jsonschema` produce no `Pub` operations in v1 + (OpenAPI has no client-streaming representation). + +## Consequences + +**Positive:** + +- The gateway is a faithful projection of the call protocol's four + operation types; no operation type is HTTP-unreachable. +- NDJSON is the natural HTTP encoding for a chunk stream (curl-able: + `echo '{"chunk":1}' | curl --data-binary @- -X POST .../publish`). +- The dispatch path is `invoke_sink()` — the same spine as `/call`'s + `invoke()`, so the shared-dispatch invariants (identity, ACL, + Internal filtering) hold by construction. + +**Negative:** + +- One more endpoint in the wire-stable gateway contract (version bump; + one-way once published). +- HTTP has no in-band abort message: a mid-stream failure is + indistinguishable from a network error to the server side (the + handler sees EOF either way). Callers needing explicit failure + semantics use the call protocol (WS channel 0). +- OQ-02 (first-line operation-naming convention) must settle before + the `/openapi.json` version bumps. + +## References + +- [http-server.md](../http-server.md) — the gateway dispatch, `/publish` + section +- [http-adapters.md](../http-adapters.md) — the gateway endpoint table +- [ADR-042](042-openapi-gateway-pattern.md) — the gateway pattern this + extends +- [ADR-045](045-to-openapi-gateway-spec-versioning.md) — version bump + mechanics +- [ADR-047](047-remove-direct-call-http-surface.md) — the gateway as + sole invoke path (now 6 endpoints) +- [ADR-023](023-operation-error-schemas.md) — error mapping +- alkcall ADR-046 (Publish Operation Type and `HandlerKind::Sink`) — + the `Pub`/`invoke_sink()`/`PublishStream` machinery this endpoint + exposes +- [open-questions.md](../open-questions.md) OQ-02 — first-line + convention, error-envelope position \ No newline at end of file diff --git a/docs/architecture/decisions/069-webtransport-out-of-scope.md b/docs/architecture/decisions/069-webtransport-out-of-scope.md new file mode 100644 index 0000000..2973b0a --- /dev/null +++ b/docs/architecture/decisions/069-webtransport-out-of-scope.md @@ -0,0 +1,102 @@ +# ADR-069: WebTransport Is Out of Scope in alkhttp + +## Status + +Accepted + +## Supersedes + +The deferral mechanics of +[ADR-044](044-defer-webtransport-browsers-use-websocket.md) (as applied +to the `alknet-http` crate): the "deferred within this crate, revives +on trigger" framing is replaced by removal from the crate's scope. The +substantive outcome ADR-044 committed — **the browser bidirectional +path is WebSocket** — stands, now realized as +[ADR-067](067-websocket-carries-channels.md). + +## Context + +The alknet design went back and forth on HTTP/3 + WebTransport +(`h3`): first-class (ADR-038), then deferred-with-revival-trigger +(ADR-044), with the ALPN-stream-proxy and bidirectional-substrate +designs parked (ADR-040, ADR-043) pending a concrete browser-side use +case (a WASM SSH/SFTP/git client reaching non-call ALPNs). + +The extraction changes the frame. alkhttp is a lean HTTP interface +crate on top of alkcall, which is transport-agnostic. WebTransport is +a *transport* concern — it involves the h3 handshake, QUIC stream +management, and TLS identity provisioning (X.509 for browsers, +[ADR-027](027-tls-identity-redesign-acme-rawkey-decoupling.md)) — all +of which live in the alknet layer, not in the HTTP interface library. +Keeping WebTransport in alkhttp would mean either shipping a transport +stack in this crate (violating the extraction's lean-crate goal) or +keeping a dormant design doc for a feature with no concrete consumer. + +Meanwhile, the concrete browser need that motivated the whole +WebTransport track is now met differently: the browser bidirectional +path is WebSocket *carrying the channels protocol* +([ADR-067](067-websocket-carries-channels.md)). A WASM SSH client in a +browser no longer needs WebTransport's multi-stream model — it opens a +data channel over the WS channels session, exactly as a Rust +consumer would. + +## Decision + +**alkhttp does not implement, feature-gate, or specc `h3`/WebTransport. +No `h3` ALPN registration, no `wtransport`/h3 dependency, no +webtransport spec document in this crate.** The `h3` ALPN handler, +the ALPN-stream-proxy, and any WebTransport relay are alknet-layer +concerns — if and when the alknet layer revives them, the alkcall ADRs +(parked in the mono-repo's history) and the alknet architecture docs +are the reference, and the alkhttp `HttpAdapter` design is +transport-agnostic enough to compose with whatever substrate alknet +provides. + +### What survives from the WebTransport work + +- **The "browser is not a peer" rationale** + ([ADR-034](034-outgoing-only-x509-and-three-peer-roles.md) §4, + amended by ADR-044 §5): stated transport-agnostically, it applies to + any browser transport — WS today, WebTransport if alknet ever + revives it. +- **Browsers require X.509** ([ADR-027](027-tls-identity-redesign-acme-rawkey-decoupling.md)): + applies to any browser-facing TLS, including the WS path's TLS. +- **The channels substrate modes** (alkcall ADR-034): the in-line mode + is implemented (TCP+TLS today, WS per + [ADR-067](067-websocket-carries-channels.md)); a native + multi-stream mode (QUIC, WebTransport) composes later without + changing the wire format or the handler experience. + +## Consequences + +**Positive:** + +- alkhttp stays lean: no h3 stack, no dormant code paths, no parked + design docs in this repo. +- The browser bidirectional story is complete without WebTransport + (ADR-067). +- Transport experimentation stays in the alknet layer where TLS, dial, + and endpoint ownership already live. + +**Negative:** + +- A browser cannot use HTTP/3-native multiplexing for alk sessions. + WebSocket over HTTP/2 (or HTTP/1.1) is the path; its framing cost + (one chunk stream over one message stream) is accepted. +- If WebTransport revives in alknet, some coordination is needed so the + alknet-side handler can present `Connection`s to alkhttp-shaped + handlers — mitigated by alkcall's `BidiStreamSource` abstraction + being exactly that seam. + +## References + +- [ADR-044](044-defer-webtransport-browsers-use-websocket.md) — the + deferral this ADR supersedes (its WS decision stands) +- [ADR-067](067-websocket-carries-channels.md) — the browser + bidirectional path as implemented +- [ADR-027](027-tls-identity-redesign-acme-rawkey-decoupling.md) — + browser TLS requirements (survives, applies to WS) +- [ADR-034](034-outgoing-only-x509-and-three-peer-roles.md) — the + browser-is-not-a-peer rationale (survives, transport-agnostic) +- The alknet mono-repo — owner of the endpoint, TLS, QUIC, and + (deferred/removed) WebTransport concerns and their ADR history \ No newline at end of file diff --git a/docs/architecture/decisions/070-from-wss-consumer-adapter.md b/docs/architecture/decisions/070-from-wss-consumer-adapter.md new file mode 100644 index 0000000..b58fee3 --- /dev/null +++ b/docs/architecture/decisions/070-from-wss-consumer-adapter.md @@ -0,0 +1,137 @@ +# ADR-070: `from_wss` — the WSS Consumer Adapter + +## Status + +Accepted + +## Context + +The alknet design named `from_wss` as a future, out-of-scope adapter +(alknet `websocket.md` §"Future"): importing a remote alk node's +operations over a WebSocket connection, mirroring `from_call`'s +pattern with WSS as the transport. It was deferred there because no +concrete consumer existed. + +Extraction changes the calculus. The browser path is now +channels-over-WebSocket ([ADR-067](067-websocket-carries-channels.md)), +which means: + +1. **The server half of WS-channels is being built anyway.** The + consumer half (`ChannelClient`-over-WS) is the mirror image sharing + the same WS↔byte-stream adapter (OQ-01) — building it later means + re-validating the adapter independently. +2. **A deployment pattern became concrete:** a Rust process behind a + restrictive network (outbound-only HTTPS/WSS allowed, no QUIC + reachability) that needs to consume another node's operations. QUIC + (`from_call` over an `alk/call` connection) and TCP+TLS + (`alk/channels`) both require reachability the deployment may not + have; WSS rides standard HTTPS infrastructure. +3. **alkcall provides both halves.** `ChannelClient` + (alkcall ADR-043) is the channels consumer; `from_call` + (alkcall ADR-028) is the same-protocol importer pattern. `from_wss` + composes them over a WSS transport. + +## Decision + +**alkhttp ships `from_wss`: a consumer adapter that connects to a +remote node's WSS endpoint, runs the channels-over-WS session, and +imports the remote node's operations as forwarding handlers — the +same-protocol importer, with WSS as the transport instead of QUIC.** + +- **Feature gate:** `wss = ["dep:tokio-tungstenite"]` (not default — + a process that never consumes over WSS should not compile a WS + client). +- **Shape:** implements `OperationAdapter` + ([ADR-017](017-call-protocol-client-and-adapter-contract.md); record: + alkcall ADR-022): + + ```rust + pub struct FromWss { + endpoint: url::Url, // wss://host/alk/channels + auth_token: Option, // injected via Capabilities at registration + namespace: Option, // local-naming sugar, as from_call's namespace_prefix + } + + #[async_trait] + impl OperationAdapter for FromWss { + async fn import(&self) -> Result, AdapterError>; + } + ``` + +- **Import flow:** dial WSS → wrap the tungstenite stream as a + `Connection` (`Connection::from_bidi`, ALPN `alk/channels`) → run + the consumer side of the channels session (alkcall `ChannelClient` + machinery: install channel 0, run the client dispatch loop) → + `services/list` + `services/schema` over channel 0 → one + forwarding `HandlerRegistration` per discovered op (provenance + `FromCall`, leaf, `Internal` by default — + [ADR-015](015-privilege-model-and-authority-context.md), + [ADR-022](022-handler-registration-provenance-and-composition-authority.md)). +- **Forwarding at call time:** each imported op's handler serializes + the call input as a `call.requested` frame, writes it (length-prefixed + JSON, alkcall ADR-014) into channel 0's write half, and correlates + `call.responded`/`call.completed`/`call.aborted` by `id` via the + pending map — identical to `from_call`'s forwarding shape, because + the protocol is the same protocol. +- **Auth:** the WSS upgrade request carries + `Authorization: Bearer `; the token comes from + `OperationContext.capabilities` at handler construction (the + no-env-vars path — + [ADR-014](014-secret-material-flow-and-capability-injection.md)), + never from `std::env::var`. +- **Relationship to the server path:** the WS↔byte-stream adapter is + shared with the server-side upgrade handler (OQ-01) — one + implementation, used in both directions. The session itself is + alkcall's (`ChannelClient` consumer half + channel 0 dispatch + loop), not forked. + +### Not in scope + +- **Reconnection policy semantics** (auto-reconnect, stale + registration invalidation): tracked in OQ-03; v1 surfaces a + connection-drop as imported-op call failures (`INTERNAL`, retryable) + and leaves policy to the assembly layer. +- **Serving WSS from the consumer side** — the producer side of a + WSS session is the `HttpAdapter` upgrade route + ([ADR-067](067-websocket-carries-channels.md)); no separate server + type. +- **`from_wss` over non-TLS `ws://`** — plaintext WS is allowed by the + underlying transport for local/test use but is not the adapter's + documented path. + +## Consequences + +**Positive:** + +- Consumers behind outbound-only HTTPS get the full operation surface + of a remote node without QUIC/TCP reachability. +- The WS↔byte-stream adapter is validated in both directions by + construction (server upgrade path + consumer path share it). +- Same-protocol import means zero translation: the remote node's ops + appear in the local registry with their real specs and error schemas. + +**Negative:** + +- One more WS client dependency in the tree when `wss` is enabled + (`tokio-tungstenite`). +- Long-lived WSS connections behind proxies/load balancers need + keepalive/timeout tuning — deployment concern, but worth documenting. +- Reconnection semantics are initially minimal (OQ-03); a consumer + wanting hot re-registration must wait for or build the policy. + +## References + +- [websocket.md](../websocket.md) — the channels-over-WS session spec + (framing shared with this adapter) +- [ADR-067](067-websocket-carries-channels.md) — the server-side + counterpart (upgrade path `/alk/channels`) +- [ADR-017](017-call-protocol-client-and-adapter-contract.md) — the + `OperationAdapter` contract +- [ADR-014](014-secret-material-flow-and-capability-injection.md) — + credential injection (the WSS bearer token path) +- [ADR-048](048-websocket-native-session-not-gateway.md) — why the + consumer speaks the native session, not the gateway shape +- alkcall ADR-022 (adapter contract), ADR-028 (`from_call` — the + same-protocol importer pattern this mirrors), ADR-043 + (`ChannelClient`), ADR-014 (framing), ADR-034/035 (channels wire) +- [open-questions.md](../open-questions.md) OQ-01, OQ-03 \ No newline at end of file diff --git a/docs/architecture/http-adapters.md b/docs/architecture/http-adapters.md new file mode 100644 index 0000000..3818199 --- /dev/null +++ b/docs/architecture/http-adapters.md @@ -0,0 +1,711 @@ +--- +status: draft +last_updated: 2026-08-27 +--- + +# HTTP Adapters — from_openapi, from_jsonschema, and to_openapi + +The OpenAPI-direction adapters plus the single-endpoint adapter: +`from_openapi` imports external HTTP APIs described by a full OpenAPI +document, `from_jsonschema` imports a single non-standard / non-OpenAPI +HTTP endpoint described by a caller-supplied `OperationSpec`, and +`to_openapi` generates an OpenAPI spec from the local registry's +`External` operations. This document covers all three, the error +fidelity (alkcall ADR-016 — Operation Error Schemas), and the +no-env-vars credential injection point. + +## What + +Three adapters, all in `alkhttp`: + +1. **`from_openapi`** — parses an OpenAPI document, constructs a + `HandlerRegistration` bundle per OpenAPI operation with a forwarding + handler that calls the external HTTP endpoint via `reqwest`, and + returns the bundles for registration in the `OperationRegistry`. The + adapter implements `OperationAdapter` (the async trait from + `alkcall::client` — alkcall ADR-022 §5, Call Protocol Client and + Adapter Contract). Provenance is `FromOpenAPI` (leaf, + `composition_authority: None`, `scoped_env: None`, `Internal` by + default — alkcall ADR-017/018). +2. **`from_jsonschema`** — registers a single HTTP endpoint as a + call-protocol operation, one at a time, for non-standard / + non-OpenAPI / basic REST endpoints that don't have a full OpenAPI + document. The caller supplies an `OperationSpec` + `HttpServiceConfig` + + path template + HTTP method; the adapter builds one + `HandlerRegistration` with a reqwest forwarding handler (the same + handler shape as `from_openapi`) and `FromJsonSchema` provenance. + Implements `OperationAdapter`. See + [ADR-066](decisions/066-from-jsonschema-as-http-adapter.md). +3. **`to_openapi`** — generates an OpenAPI document from the local + registry's `External` operations. A pure projection: it consumes the + registry, it does not produce entries for it (alkcall ADR-022 §5 — + the `to_*` adapters are outbound projections, not `OperationAdapter` + implementations). Served at `GET /openapi.json` by the HTTP server. + +### from_openapi + +```rust +pub struct FromOpenAPI { + spec: OpenAPISpec, + config: HttpServiceConfig, +} + +#[async_trait] +impl OperationAdapter for FromOpenAPI { + async fn import(&self) -> Result, AdapterError>; +} +``` + +#### Type definitions + +```rust +/// A parsed OpenAPI document. The internal representation is +/// `serde_json::Value`-based (ADR-051 §Assumptions #1) — both JSON and +/// YAML parse paths produce the same `serde_json::Value` tree, then feed +/// the existing `from_value` constructor. A future swap to +/// `openapiv3::OpenApi` is a two-way door: both JSON and YAML constructors +/// adapt in lockstep, since the constructor is the adapter between wire +/// format and internal type. The one-way constraint is that +/// `from_openapi` accepts a standard OpenAPI 3.x JSON/YAML doc and +/// `to_openapi` produces one. Both directions share the same Rust type, +/// but not the same document shape: `from_openapi` consumes traditional +/// per-operation-paths docs (one path per operation), while `to_openapi` +/// produces the 6-endpoint gateway doc (ADR-042, extended with `/publish` +/// by ADR-068). The type is shared; the shape is not. +/// +/// Input formats (ADR-051): `from_openapi` accepts both JSON and YAML. +/// JSON is parsed via `serde_json`; YAML via `yaml_serde` (the maintained +/// fork of the deprecated `serde_yaml`). Both paths produce the same +/// `serde_json::Value`-based internal type — there is one +/// `OpenAPISpec`, not a JSON and a YAML variant. `from_str` detects +/// format by trying JSON first and falling back to YAML (defensive +/// default — ADR-051 §2: JSON's stricter grammar is immune to any +/// YAML-specific type interpretation, present or future; with +/// `yaml_serde` 0.10.x's YAML 1.2 core schema the coercion the original +/// rationale cited is not present, but JSON-first locks the contract +/// against a future YAML-parser swap). +pub struct OpenAPISpec { + pub info: OpenAPIInfo, + pub paths: BTreeMap, + pub components: Option, + // ... OpenAPI 3.x fields as needed +} + +impl OpenAPISpec { + pub fn from_json(doc: &str) -> Result; // JSON input + pub fn from_yaml(doc: &str) -> Result; // YAML input + pub fn from_str(doc: &str) -> Result; // format-detecting (JSON-first, YAML-fallback — ADR-051 §2) + pub fn from_value(raw: Value) -> Result; // pre-parsed serde_json::Value +} + +/// Configuration for an HTTP-backed adapter (`from_openapi`). Carries +/// the base URL, auth credentials (from `Capabilities` at registration, +/// not env vars — the no-env-vars invariant), and optional headers. The +/// `auth` field is the auth scheme the external API expects (bearer, +/// apiKey, basic); the credential itself is read from +/// `OperationContext.capabilities` at call time, not stored here. +pub struct HttpServiceConfig { + pub namespace: String, + pub base_url: String, + pub auth: Option, + pub default_headers: HashMap, +} + +pub enum HttpAuthScheme { + Bearer, // Authorization: Bearer + ApiKey { header_name: String }, // e.g., X-API-Key: + Basic, // Authorization: Basic +} +``` + +The adapter: + +1. Parses the OpenAPI document (`OpenAPISpec` — `paths`, `components`, + `$ref` resolution). Accepts JSON or YAML (ADR-051 — JSON via + `serde_json`, YAML via `yaml_serde`; `from_str` detects format + JSON-first/YAML-fallback, a defensive default — ADR-051 §2). On parse + failure, returns `AdapterError::SchemaParse`. The TS prior art + (`@alkdev/operations/src/from_openapi.ts`) shows the parsing patterns: + `resolveRef` for `$ref`, `resolveRefsRecursive` for nested refs, + `buildInputSchema` (parameters + request body → input JSON Schema), + `buildOutputSchema` (200/201 response → output JSON Schema), + `detectOperationType` (SSE response → `Sub`, GET → `Query`, + else `Mutation`). Pub ops are not produced by `from_openapi` — + OpenAPI has no representation for producer→consumer streaming in v1. +2. For each `(path, method, operation)` in `spec.paths`, constructs a + `HandlerRegistration`: + - `spec.name` = the `operationId` (or a generated + `${method}_${path_parts}` name if `operationId` is absent — same + normalization as the TS `normalizeOperationId`). + - `spec.namespace` = the `config.namespace` (the importing + deployment's name for the service, not the OpenAPI doc's `info.title`). + - `spec.op_type` = `Query` / `Mutation` / `Sub` (detected as `Sub` + from the method + response content type, same as TS). + - `spec.visibility` = `Internal` (adapter-registered ops are + composition material, not directly callable from the wire — + alkcall ADR-017). + - `spec.input_schema` / `output_schema` = the JSON Schemas built + from the OpenAPI parameters/responses. + - `spec.error_schemas` = the `ErrorDefinition`s built from the + non-2xx OpenAPI responses (alkcall ADR-016 §5 — see Error + Fidelity below). + - `spec.access_control` = `AccessControl::default()` (the adapter + doesn't declare scopes; the composing handler that reaches the + imported op gates access). + - `handler` = a forwarding handler (see Forwarding Handler below). + - `provenance` = `FromOpenAPI`, `composition_authority: None`, + `scoped_env: None` (leaf — alkcall ADR-018). + - `capabilities` = the credentials the forwarding handler needs (the + bearer token / API key for the external HTTP endpoint, injected by + the assembly layer at registration — see No-Env-Vars below). +3. Returns the bundles. The caller (the assembly layer) registers them + in the `OperationRegistry`. + +### Forwarding handler + +The forwarding handler is stored in the `HandlerRegistration` as a +`HandlerKind` (alkcall ADR-021). At call time, it: + +1. Reads the call input (`serde_json::Value`). +2. Builds the outbound HTTP request: + - URL path: substitutes path parameters (`{id}` → input value), + appends query parameters from input fields not in the path. + - Method: the OpenAPI operation's method. + - Headers: `Content-Type: application/json` + the auth header built + from `context.capabilities` (see No-Env-Vars below). + - Body: the `body` field of the input (for `Mutation`/`Sub`). +3. Sends the request via the shared HTTP client (see HTTP Client + below). +4. For a `Query`/`Mutation`: parses the response body (JSON, text, or + binary — same content-type branching as the TS `createHTTPOperation`), + wraps it in a `ResponseEnvelope`, returns. Registered as + `HandlerKind::Once` — a `Handler` returning a single + `ResponseEnvelope`. +5. For a `Sub` (`text/event-stream` response): streams + `call.responded` events as the SSE chunks arrive (same SSE parsing as + the TS `parseSSEFrames`), then the stream ends on SSE close (which + becomes `call.completed` on the wire). Registered as + `HandlerKind::Stream` — a `StreamingHandler` returning a + `BoxStream` (alkcall ADR-021). Each SSE `data:` + frame becomes a `ResponseEnvelope::ok()`; an HTTP error (non-2xx) + becomes a single `ResponseEnvelope::error()` and ends the stream. +6. On HTTP error (non-2xx): maps to the declared `ErrorDefinition` by + HTTP status code (see Error Fidelity below), returns a `CallError`. + +The handler is opaque to `alkcall`'s `CallAdapter` — it's a +`HandlerKind` the registry dispatches (via `invoke()` for `Once`, +`invoke_streaming()` for `Stream`). `alkcall` never sees `reqwest`. + +### HTTP client (reqwest) + +`alkhttp` maintains a shared HTTP client, constructed once and reused +across all `from_openapi`/`from_mcp` forwarding handlers. The client owns +connection pooling, keep-alive, TLS, and a retry stack. The shared type is +`reqwest_middleware::ClientWithMiddleware`, not a bare `reqwest::Client` — +both retry and Retry-After are middleware on the stack, and middleware +requires the `ClientWithMiddleware` wrapper. + +The middleware stack has two layers: + +1. **`RetryTransientMiddleware`** (from `reqwest-retry`) — exponential + backoff on transient failures (connection errors, 5xx). The "retry N + times with increasing intervals" part. Configured via an + `ExponentialBackoff` policy at client construction. +2. **Inlined `RetryAfterMiddleware`** — parses the `Retry-After` header + on 429/503 and sleeps before the next request to that URL. The + "respect what the server told you" part. Inlined (MIT, ~50 lines of + real logic) from `melotic/reqwest-retry-after`, not pulled as a + dependency: the crate is complementary to `reqwest-retry` (whose + default strategy does not honor `Retry-After`), and inlining lets + the upstream's unbounded `HashMap` storage be + bounded for a long-running process. + +Pooling, keep-alive, and TLS come from `reqwest::ClientBuilder` defaults; +outbound TLS uses the system trust store (standard HTTPS to external APIs +like OpenAI, Anthropic). Custom CA bundle + client certs are an optional +config for self-hosted API gateways (two-way-door implementation detail; +the credential comes from `Capabilities`, the TLS trust comes from the +system). + +Credential injection happens per-request (from +`OperationContext.capabilities`), not at client construction — the client +is shared across all operations, the credentials are per-call. + +Hot-reload of the pooling/retry config is **rebuild-and-swap**: a config +change rebuilds the `ClientWithMiddleware` and swaps it via `ArcSwap` +(the same pattern `ConfigIdentityProvider` uses for its +`ArcSwap` reload — see the alkcall crate's +`docs/architecture/decisions/006-authcontext-structure.md` and +`docs/architecture/decisions/025-peerentry-and-identity-id-decoupling.md`). +A rebuild drops the connection pool / keep-alive state, which is +acceptable — a config change wanting a fresh pool is the case that +triggers it. The retry policy is baked into the middleware at +`ClientBuilder::build()` time; live policy mutation is not supported by +`reqwest-retry`, so cheap per-policy updates are not part of the model. + +The exact pooling/retry config (pool size, retry count, timeout +defaults, hot-reloadability via `DynamicConfig`) is a two-way-door +implementation detail (OQ-40, now resolved); the one-way constraint is +that `alkhttp` owns its HTTP client (no env-var-based client config, +no shared global client). + +**Downstream layering boundary.** The agent crate's provider SSE +normalization (replicating the solid part of aisdk's pattern — the +Vercel-UI-message normalization that maps different providers' SSE to a +common shape) sits on top of this `ClientWithMiddleware`: it consumes the +`reqwest::Response` stream the forwarding handler produces and emits +`call.responded` events. It does not replace the client or own +transport/pooling/retry. `alkhttp` owns transport; the agent crate +owns provider-specific SSE → Vercel-UI-message mapping. The aisdk +`core/client.rs` reference for HTTP client construction is *not* carried +forward — its env-var config and hand-rolled retry are the anti-patterns +discarded in favor of the middleware stack above. The +`@alkdev/operations/src/from_openapi.ts` SSE *normalization* pattern is +separate and stays referenced in the Forwarding Handler section above +(the `parseSSEFrames`, `createHTTPOperation`, content-type branching +patterns). + +### No-Env-Vars credential injection + +The forwarding handler is the **credential injection point** for the +no-env-vars architecture. The handler reads +`context.capabilities.get("")` (e.g., `"openai"`, `"vastai"`, +`"github"`), extracts the credential, and injects it into the outbound +HTTP request: + +- Bearer token → `Authorization: Bearer `. +- API key → the header the OpenAPI spec declares (e.g., `X-API-Key: + `, or `Authorization: ApiKey ` — the `HTTPServiceConfig.auth` + in the TS prior art shows the three auth types: `bearer`, `apiKey`, + `basic`). +- Basic auth → `Authorization: Basic `. + +The credential comes from `Capabilities`, which was populated by the +dispatch path from the `HandlerRegistration.capabilities` bundle +(alkcall ADR-018 §6), which was populated by the assembly layer from the +vault ([ADR-014](decisions/014-secret-material-flow-and-capability-injection.md)). +The handler never reads `std::env::var`. This is the +spec-level invariant: no handler reads outbound credentials from any +source other than `OperationContext.capabilities`. See +[overview.md](overview.md) and the alkcall crate's +`docs/architecture/client-and-adapters.md`. + +### from_jsonschema + +`from_jsonschema` registers a single HTTP endpoint as a call-protocol +operation, one at a time. It is functionally similar to `from_openapi` +but for one endpoint instead of a full OpenAPI document — for +non-standard, non-OpenAPI, or basic REST endpoints that don't have a +`paths` object, an `operationId`, or `components`. The caller supplies +the schema directly; the adapter builds a reqwest forwarding handler +identical in shape to `from_openapi`'s. See +[ADR-066](decisions/066-from-jsonschema-as-http-adapter.md). + +```rust +pub struct FromJsonSchema { + spec: OperationSpec, + config: HttpServiceConfig, + path_template: String, + method: String, + http_client: Arc, +} + +#[async_trait] +impl OperationAdapter for FromJsonSchema { + async fn import(&self) -> Result, AdapterError>; +} +``` + +The adapter: + +1. Takes an `OperationSpec` (name, op type, input/output JSON Schema, + `error_schemas`, `access_control`, `visibility`), an + `HttpServiceConfig` (base URL, auth scheme, default headers — the + same config type `from_openapi` uses), a path template + (e.g. `/users/{id}/posts`), and an HTTP method (e.g. `GET`). +2. Builds one `HandlerRegistration`: + - `spec` = the caller-supplied `OperationSpec` (the caller already + has the JSON Schemas; no parsing needed). + - `handler` = a reqwest forwarding handler, identical in shape to + `from_openapi`'s: builds the HTTP request (path-template + substitution, query params, body), injects credentials from + `context.capabilities`, sends via the shared HTTP client, parses + the response (JSON / text / binary — same content-type branching). + For `Sub` op type, registers a `StreamingHandler` + (alkcall ADR-021) expecting `text/event-stream`. + - `provenance` = `FromJsonSchema` (leaf, `composition_authority: None`, + `scoped_env: None` — alkcall ADR-018). + - `capabilities` = the credentials the forwarding handler needs + (same no-env-vars path as `from_openapi`). +3. Returns the single bundle. The caller registers it in the + `OperationRegistry`. + +#### Relationship to from_openapi + +`from_jsonschema` is functionally similar to `from_openapi` but for one +endpoint instead of a full OpenAPI document. The two adapters share the +forwarding-handler implementation, the credential injection path, the +error-fidelity rule (`HTTP_` prefix, alkcall ADR-016), the +streaming shape (alkcall ADR-021), and the no-env-vars invariant +([ADR-014](decisions/014-secret-material-flow-and-capability-injection.md)). +The difference is purely the input shape: a full document vs. a single +endpoint. See +[ADR-066](decisions/066-from-jsonschema-as-http-adapter.md) +§"Relationship to `from_openapi`" for the comparison table. + +#### Origin (ADR-066) + +`from_jsonschema` was originally placed in the call crate +(alkcall ADR-022 §5) as a schema-only adapter with a +`NOT_FOUND`-returning placeholder handler — broken, because an op in the +registry needs a real handler. +[ADR-066](decisions/066-from-jsonschema-as-http-adapter.md) moved +it to `alkhttp` and gave it a real reqwest forwarding handler. The +`FromJsonSchema` provenance variant stays in `alkcall` +(`OperationProvenance`, in `alkcall::registry` — alkcall ADR-027 records +the move from the call-crate side); only the adapter implementation +moved. See +[ADR-066](decisions/066-from-jsonschema-as-http-adapter.md) for the +full rationale (why the placeholder was broken, why the "schema-only" +concept conflated two things, why it was mispaced in the call crate). + +### to_openapi + +```rust +pub fn to_openapi(registry: &OperationRegistry) -> OpenAPISpec; +``` + +`to_openapi` generates an OpenAPI document with a **fixed gateway +endpoint set** that gates access to the full operation registry — not +one path per operation. This is the OpenAPI gateway pattern (ADR-042): +the same principle as the MCP gateway (ADR-041) applied to OpenAPI. The +external client (a code generator, a human developer, a `fetch`-based +client) calls `/search` to discover operations, `/schema` to learn an +operation's input shape, `/call` (or `/batch`, `/subscribe`, or +`/publish`) to invoke. See +[ADR-042](decisions/042-openapi-gateway-pattern.md) for the +rationale (the flat→structured split problem, the per-caller API +surface problem). + +#### The gateway endpoint set + +`to_openapi` generates 6 fixed endpoints — the original five from +[ADR-042](decisions/042-openapi-gateway-pattern.md) plus `/publish` +([ADR-068](decisions/068-gateway-publish-endpoint.md)): + +| OpenAPI path | Call protocol | HTTP method | Purpose | +|--------------|--------------|-------------|---------| +| `/search` | `services/list` | `GET` | List/search operations (AccessControl-filtered). Names + descriptions. | +| `/schema` | `services/schema` | `GET` | Get an operation's full `OperationSpec`. | +| `/call` | `call.requested` (Query/Mutation) | `POST` | Invoke an operation. Flat JSON body `{ operation, input }`. | +| `/batch` | multiple `call.requested` | `POST` | Invoke multiple operations. Array of `{ operation, input }`. | +| `/subscribe` | `call.requested` (Sub) | `POST` (SSE) | Invoke a streaming operation. Body `{ operation, input }` (same shape as `/call`); response is `text/event-stream`. | +| `/publish` | `call.requested` (Pub) | `POST` (NDJSON) | Publish to a Pub operation. Request body is newline-delimited JSON — each line is one published chunk, streamed to the operation's `HandlerKind::Sink` handler (alkcall ADR-046; [ADR-068](decisions/068-gateway-publish-endpoint.md)). | + +The input is always a flat JSON body — no path/query/body split to +reverse-engineer. JSON Schema for the input/output is already in the +`OperationSpec`; the gateway wraps it in OpenAPI's schema format without +splitting parameters. + +`/subscribe` and `/publish` are the two endpoints the MCP gateway +excludes ([ADR-041](decisions/041-mcp-tool-gateway-pattern.md) — +MCP tool calls are request/response; the tool-gateway surface has +neither a streaming nor a client-publish shape). OpenAPI/SSE supports +streaming; the gateway's `/subscribe` uses the same SSE projection +[ADR-036](decisions/036-http-to-call-operation-mapping.md) +describes — `call.responded` → SSE `data:` frames, `call.completed` → +stream close. `/publish` inverts the direction: the HTTP caller's +newline-delimited JSON lines become `call.published` chunks on the call +protocol ([ADR-068](decisions/068-gateway-publish-endpoint.md)). + +#### Per-caller API surface + +The `/search` endpoint's results are `AccessControl::check(identity)`- +filtered — the client sees only the operations it is authorized to call. +The generated OpenAPI doc describes the 6 gateway endpoints (stable, +same for every caller); the per-caller operation surface is discovered +through `/search`, not preloaded into the doc. This is the key +advantage over a traditional per-operation-paths OpenAPI doc: the +per-caller API surface is the default (the Gitea failure mode — dumping +admin ops to every caller — is structurally impossible). See +[ADR-042](decisions/042-openapi-gateway-pattern.md) §3. + +#### Pure projection + +`to_openapi` is a pure projection — it consumes the registry and +produces a spec. It does not modify the registry; it does not register +operations; it is not an `OperationAdapter`. The HTTP server serves the +generated spec at `GET /openapi.json` (or a configured path). + +#### Traditional per-operation-paths projection (additive) + +A deployment that wants a traditional REST OpenAPI doc (per-operation +paths with split parameters) can build it as a separate projection with +HTTP-specific metadata (which fields are path params, etc.). The +gateway pattern is the default `to_openapi` projection; the traditional +projection is additive, not a replacement. See +[ADR-042](decisions/042-openapi-gateway-pattern.md) §5. + +#### Shared dispatch spine with `to_mcp` + +`to_openapi`'s `/call` endpoint and `to_mcp`'s `call` tool share the +same dispatch spine (resolve identity → build `OperationContext` → +`OperationRegistry::invoke()` → map `ResponseEnvelope`). The +wire-framing, discovery, streaming, and server-integration layers are +per-gateway. See [http-mcp.md](http-mcp.md) §"Shared dispatch spine +with `to_openapi`" and +`/workspace/@alkdev/alknet/docs/research/alknet-http-gateway-factoring/findings.md` +for the factoring recommendation (thin shared struct, not a trait). + +### Error Fidelity (alkcall ADR-016) + +`from_openapi` maps OpenAPI non-2xx response status codes to +`ErrorDefinition`s (alkcall ADR-016 §5). The normative rule (review +#002 W20): `from_openapi` must not produce error codes that collide +with the six protocol-level codes (`NOT_FOUND`, `FORBIDDEN`, +`INVALID_INPUT`, `INVALID_OPERATION_TYPE`, `INTERNAL`, `TIMEOUT`). The +adapter prefixes imported error codes with `HTTP_` and the status +number: + +```rust +// OpenAPI: 404: { schema: NotFoundError } +// → ErrorDefinition { code: "HTTP_404", http_status: Some(404), schema: NotFoundError } +``` + +`to_openapi` projects `error_schemas` to the gateway endpoint's +response definitions. The `/call` endpoint's responses include the +operation-level errors (mapped by `http_status`), plus the protocol- +level errors: + +```yaml +# /call endpoint responses +responses: + '200': { schema: } + '400': { schema: } + '401': { schema: } + '403': { schema: } + '404': { schema: } + '422': { schema: } + '429': { schema: } + '500': { schema: } + '504': { schema: } +``` + +The operation-level errors (with `http_status`) are surfaced on the +`/call` endpoint's response — the gateway propagates the called +operation's `error_schemas` as response definitions. This makes the +adapter contract from alkcall ADR-022 faithful on the error axis — no +silent dropping of error contracts. See alkcall ADR-016. + +## Why + +`from_openapi` is how the alk stack composes external HTTP APIs (OpenAI, +Anthropic, vast.ai, GitHub) into the call protocol. An operation +imported via `from_openapi` is a first-class operation: it has a spec, +it's discoverable via `services/list`, it can be composed by handlers, +its errors are typed. The agent crate's LLM provider calls go through +`from_openapi`-imported operations — that's how the no-env-vars +invariant makes aisdk's env-var reads unreachable. + +`from_jsonschema` fills the gap that `from_openapi` can't: endpoints +that have no OpenAPI document. A non-standard REST endpoint, a basic +internal API, or a third-party service with only a JSON Schema +description can be registered as a call-protocol operation one at a +time, with the same reqwest forwarding handler and the same +no-env-vars credential path. The caller supplies the schema; the +adapter supplies the handler. See +[ADR-066](decisions/066-from-jsonschema-as-http-adapter.md). + +`to_openapi` is how external systems discover the alk stack's operation +surface. A client generator, a human developer, or a `fetch`-based +client reads the OpenAPI doc to learn the gateway's shape (6 fixed +endpoints), then calls `/search` to discover what *it* can call +(per-caller, AccessControl-filtered) and `/schema` to learn an +operation's input shape. The gateway pattern avoids the flat→structured +split that a traditional per-operation-paths projection would require, +and makes the per-caller API surface the default (the Gitea failure +mode — dumping admin ops to every caller — is structurally impossible). +See [ADR-042](decisions/042-openapi-gateway-pattern.md). The +generated spec is a compatibility contract (alkcall ADR-022 +Consequences) — once published, the 6-endpoint gateway shape is +one-way. + +## Constraints + +- **`from_openapi`/`from_mcp` handlers read credentials from + `OperationContext.capabilities`, not `std::env::var`.** This is the + no-env-vars invariant + ([ADR-014](decisions/014-secret-material-flow-and-capability-injection.md)). + The handler implementations are verified against this invariant. + `from_jsonschema` shares this invariant — same handler shape, same + credential path ([ADR-066](decisions/066-from-jsonschema-as-http-adapter.md)). +- **`from_openapi`-registered ops are `Internal` by default.** They are + composition material, not directly callable from the wire (alkcall + ADR-017). The handler that composes them is `External`. + `from_jsonschema` ops are `Internal` by default for the same reason + ([ADR-066](decisions/066-from-jsonschema-as-http-adapter.md)). +- **`from_openapi` error codes are prefixed `HTTP_`.** No + collision with protocol-level codes (alkcall ADR-016, review #002 + W20). `from_jsonschema` shares this rule + ([ADR-066](decisions/066-from-jsonschema-as-http-adapter.md)). +- **`from_openapi` accepts JSON and YAML; `from_str` detects format + JSON-first.** JSON-first is a defensive default (ADR-051 §2 as + amended): JSON's stricter grammar is immune to any YAML-specific type + interpretation. With `yaml_serde` 0.10.x (YAML 1.2 core schema) the + coercion the original rationale cited is not present, but JSON-first + locks the contract against a future YAML-parser swap. `from_str` tries + JSON first, falls back to YAML only if JSON parse fails. + `from_json`/`from_yaml` are the explicit constructors for callers that + know the format. +- **`to_openapi` is a pure projection.** It consumes the registry, does + not produce entries for it. Not an `OperationAdapter`. +- **`to_openapi` output is JSON.** The published gateway doc is served at + `GET /openapi.json`. YAML output is out of scope (ADR-051 §4); the gap + this fills is on the consume side (importing external YAML schemas), + not the publish side. +- **Published `to_openapi` specs are compatibility contracts.** The + generated gateway doc carries `info.version` (semver) tracking the + **gateway endpoint contract**, not the operation set — per-caller + operation changes (add/remove/modify, schema changes) do not bump + the version (the operation set is discovered via `/search`, not + preloaded into the doc). Consumers detect breaking changes via the + major version (alkcall ADR-022 Consequences, + [ADR-045](decisions/045-to-openapi-gateway-spec-versioning.md), + resolves OQ-39). +- **`alkhttp` owns its HTTP client.** Shared across all forwarding + handlers, constructed once. The shared type is + `reqwest_middleware::ClientWithMiddleware` (middleware stack: + `RetryTransientMiddleware` + inlined `RetryAfterMiddleware`). No + env-var-based client config. Pooling/retry config is a two-way door, + resolved in OQ-40. +- **TLS for outbound calls uses the system trust store by default.** + Standard HTTPS to external APIs (OpenAI, Anthropic). Custom CA bundle + + client certs are an optional config for self-hosted API gateways. + This is a two-way-door implementation detail; the credential (API + key/token) comes from `Capabilities`, the TLS trust comes from the + system. + +## Design Decisions + +| Decision | ADR | Summary | +|----------|-----|---------| +| `from_openapi` is an `OperationAdapter` | alkcall ADR-022 (Call Protocol Client and Adapter Contract) | Async trait (`alkcall::client`); produces `HandlerRegistration` bundles. ~~`from_jsonschema` clause superseded by ADR-066~~ | +| `from_jsonschema` as HTTP-backed single-endpoint adapter in `alkhttp` | [ADR-066](decisions/066-from-jsonschema-as-http-adapter.md) | Moved `from_jsonschema` from the call crate (broken schema-only placeholder — the call-crate side of the move is alkcall ADR-027) to `alkhttp` as a real reqwest-backed single-endpoint adapter; `FromJsonSchema` provenance stays in `alkcall` as a leaf | +| `to_openapi` is a projection, not an adapter | alkcall ADR-022 (Call Protocol Client and Adapter Contract) | Consumes the registry, doesn't produce entries | +| Adapter-registered ops are `Internal` | alkcall ADR-017 (Privilege Model and Authority Context) | `from_openapi` ops are composition material | +| `from_openapi` provenance is a leaf | alkcall ADR-018 (Handler Registration, Provenance, and Composition Authority) | `composition_authority: None`, `scoped_env: None` | +| Error fidelity (`HTTP_` codes) | alkcall ADR-016 (Operation Error Schemas) | No collision with protocol codes; `to_openapi` projects back | +| No-env-vars credential injection | [ADR-014](decisions/014-secret-material-flow-and-capability-injection.md) | Handler reads `context.capabilities`, not env vars | +| HTTP path = operation path (~~direct-call surface~~) | [ADR-036](decisions/036-http-to-call-operation-mapping.md) → superseded by [ADR-047](decisions/047-remove-direct-call-http-surface.md) | ~~`POST /{service}/{op}` → `call.requested`~~ — removed; the gateway `/call` with `{ operation, input }` is the sole invoke path; `to_openapi` describes the gateway, not a per-operation surface | +| `to_openapi` gateway pattern | [ADR-042](decisions/042-openapi-gateway-pattern.md) | 6 fixed gateway endpoints (search/schema/call/batch/subscribe/publish — `/publish` per [ADR-068](decisions/068-gateway-publish-endpoint.md)), not one path per operation; per-caller AccessControl-filtered. Supersedes ADR-036's original `to_openapi` "paths mirror `/{service}/{op}`" clause | +| `to_openapi` published-spec versioning | [ADR-045](decisions/045-to-openapi-gateway-spec-versioning.md) | `info.version` semver tracks the gateway endpoint contract, not the operation set; consumers detect breaking changes via the major version | +| Streaming handler for subscriptions | alkcall ADR-021 (Streaming Handler for Subscription Operations) | `from_openapi` / `from_jsonschema` `Sub` ops register a `StreamingHandler` (`HandlerKind::Stream`); SSE response → `BoxStream`; `Query`/`Mutation` stay `HandlerKind::Once` | +| YAML input + JSON-first format detection | [ADR-051](decisions/051-yaml-input-for-from-openapi.md) | `from_openapi` accepts JSON and YAML (`from_json`/`from_yaml`/`from_str`); `from_str` is JSON-first/YAML-fallback (defensive default, §2 amended — `yaml_serde` 0.10.x is YAML 1.2, not 1.1; JSON-first locks the contract against a future parser swap); YAML dep is `yaml_serde`; `to_openapi` output stays JSON (out of scope, §4) | + +## Open Questions + +See [open-questions.md](open-questions.md) for full details. + +- **OQ-39** (resolved): `to_openapi` published-spec versioning — + resolved by + [ADR-045](decisions/045-to-openapi-gateway-spec-versioning.md): + `info.version` semver tracks the gateway endpoint contract (major = + breaking gateway change, minor = additive, patch = wording); the + per-caller operation set is discovered via `/search` and does not bump + the version. The additive traditional per-operation-paths projection + ([ADR-042](decisions/042-openapi-gateway-pattern.md) §5) versions + independently, out of scope. +- **OQ-40** (resolved): reqwest client config and connection pooling — + `ClientWithMiddleware` + `RetryTransientMiddleware` + inlined + `RetryAfterMiddleware`; rebuild-and-swap hot-reload; per-request + credential injection. Two-way-door config shape, now resolved. + +## References + +- alkcall ADR-022 (Call Protocol Client and Adapter Contract) — the + `OperationAdapter` trait (in `alkcall::client`), `to_*` are + projections +- [ADR-066](decisions/066-from-jsonschema-as-http-adapter.md) — + `from_jsonschema` as HTTP-backed single-endpoint adapter in + `alkhttp` (supersedes alkcall ADR-022 §5's `from_jsonschema` clause; + the call-crate-side record of the move is alkcall ADR-027) +- alkcall ADR-016 (Operation Error Schemas) — error fidelity, + `HTTP_` prefix rule +- [overview.md](overview.md) — adapter location map, no-env-vars + invariant +- the alkcall crate's `docs/architecture/client-and-adapters.md` — + `OperationAdapter` trait, `AdapterError` variants (OQ-26), no-env-vars + invariant +- `/workspace/@alkdev/operations/src/from_openapi.ts` — TypeScript prior + art (parsing, SSE, auth headers, `createHTTPOperation`, + `parseSSEFrames` — the SSE normalization patterns, not the client + construction) +- `reqwest-retry` crate (https://docs.rs/reqwest-retry/) — + `RetryTransientMiddleware` / `ExponentialBackoff` retry policy +- `melotic/reqwest-retry-after` + (https://github.com/melotic/reqwest-retry-after) — `RetryAfterMiddleware` + source (MIT, inlined, not a dependency) + +## Port notes + +Corrections applied during the alknet → alkhttp port, beyond mechanical +crate renames (`alknet-http` → `alkhttp`; `alknet-call`/`alknet-core` → +`alkcall`, with the adapter contract in `alkcall::client`, registry +types in `alkcall::registry`, core in `alkcall::core`): + +1. **Gateway is now 6 endpoints.** `/publish` (POST, Pub operations; + request body streamed as newline-delimited JSON, each line one + published chunk) added per alkhttp ADR-068 + (`decisions/068-gateway-publish-endpoint.md` — slug assumed; the ADR + file does not exist yet). All "5 fixed endpoints" tables/phrasing + updated to 6; the MCP-gateway exclusion now covers both `/subscribe` + and `/publish` (the "one endpoint the MCP gateway excludes" claim + from the source extended accordingly). +2. **`OperationType::Subscription` renamed `Sub`** (alkcall rename); + `OperationType::Pub` exists (producer→consumer streaming, + `HandlerKind::Sink`, wire event `call.published` — alkcall ADR-046). + The Query/Mutation/Sub detection mapping from OpenAPI is kept, with a + one-line note that Pub ops are not produced by `from_openapi` (no + OpenAPI representation in v1). The `/publish` gateway row is the only + Pub surface in this doc. +3. **ADR cross-reference remapping.** Call-protocol-internal decisions + now live in the alkcall crate and are cited textually (no relative + links across crates): old ADR-017 adapter contract → alkcall ADR-022; + old ADR-022 handler registration → alkcall ADR-018 (§6 mapping + preserved); old ADR-023 error schemas → alkcall ADR-016 (§5 mapping + preserved); old ADR-015 privilege model → alkcall ADR-017; old + ADR-049 streaming handler → alkcall ADR-021; Pub/`HandlerKind::Sink` + → alkcall ADR-046; `from_jsonschema` provenance → alkcall ADR-027. + alkhttp-owned ADRs keep their numbers and are linked relatively: + 014, 036, 041, 042, 045, 047, 051, 066 (ADR-036's port to alkhttp is + assumed — it is an HTTP-surface decision, consistent with the + other gateway/server ADRs). +4. **Old ADR-035 reference (hot-reload pattern) had no verified + mapping.** The source cited alknet ADR-035 (Concrete Persistence + Adapter Shapes) for the `ConfigIdentityProvider` `ArcSwap` + rebuild-and-swap pattern; that ADR has no direct alkcall/alkhttp + counterpart. Replaced with a textual pointer to the alkcall crate's + ADR-006 (AuthContext Structure — documents the + `ArcSwap` reload) and ADR-025 (PeerEntry and + Identity.id Decoupling — introduces `ConfigIdentityProvider`). + Flagged as an inference, not a verified mapping. +5. **Producer/consumer terminology.** Remaining "server"/"client" + wording is deliberately retained where OpenAPI's inherent + client/server directionality is meant (that directionality is a + property of the OpenAPI protocol, not the call protocol) and for the + HTTP server / external HTTP API server (transport roles, not + call-protocol roles). No call-protocol server/client framing remains. +6. **Links.** `../../decisions/` → `decisions/`; + `../../open-questions.md` → `open-questions.md` (sibling, pending + port); `../call/client-and-adapters.md` → textual reference to the + alkcall crate's `docs/architecture/client-and-adapters.md`; the + mono-repo research findings path is kept as a textual absolute path. + `overview.md` and `http-mcp.md` are sibling links to docs pending + port. +7. **WebTransport/h3.** The source document contained no h3/WebTransport + references, so no "out of scope in alkhttp (ADR-069)" replacements + were needed. +8. **Project naming.** "how alknet composes/discover the alknet + operation surface" → "the alk stack" in the Why section prose. + Everything else (decision content, structure, section order, + rationale) is preserved verbatim from the source. \ No newline at end of file diff --git a/docs/architecture/http-mcp.md b/docs/architecture/http-mcp.md new file mode 100644 index 0000000..737ef1a --- /dev/null +++ b/docs/architecture/http-mcp.md @@ -0,0 +1,481 @@ +--- +status: draft +last_updated: 2026-08-27 +--- + +# HTTP MCP — from_mcp and to_mcp + +The MCP-direction adapters (feature-gated behind `mcp`): `from_mcp` +imports remote MCP tools as call-protocol operations over streamable +HTTP (reqwest client), and `to_mcp` exposes local operations as MCP +tools over streamable HTTP (axum server). This document covers both, the +rmcp integration, and the stdio exclusion (ADR-037). + +## What + +Two adapters, both in `alkhttp`, both behind the `mcp` feature gate: + +1. **`from_mcp`** — discovers remote MCP tools via the MCP + `tools/list` call over streamable HTTP, and registers each as a + `HandlerRegistration` bundle with a forwarding handler that calls the + remote tool via `tools/call`. Uses rmcp's + `StreamableHttpClientTransport` (reqwest-based). Provenance is + `FromMCP` (leaf, `composition_authority: None`, `scoped_env: None`, + `Internal` by default — alkcall ADR-017/018). Implements + `OperationAdapter` (the async trait from `alkcall::client`). +2. **`to_mcp`** — exposes the local registry's `External` operations as + MCP tools over streamable HTTP, using rmcp's `StreamableHttpService` + (an axum-compatible tower service). An external MCP client (an editor, + an AI tool) discovers and calls the local deployment's operations + through the MCP protocol. A pure projection (consumes the registry, + does not produce entries — alkcall ADR-022 §5). + +### Streamable HTTP only (ADR-037) + +MCP defines two transports: streamable HTTP and stdio. **alkhttp +supports only streamable HTTP.** Stdio is not built — it is the spawn- +arbitrary-executable RCE vector that the rest of the architecture is +designed to avoid (ADR-037). The `mcp` feature gate pulls in rmcp with +the streamable HTTP transport features only; the stdio transport +(`transport-child-process`) is not a dependency, not optional, not +behind a separate feature. + +If an operator wants a stdio-only MCP server, they run a small +streamable-HTTP-to-stdio bridge themselves, outside alkhttp. The bridge +is where the RCE risk lives, explicitly in the operator's hands. See +[ADR-037](decisions/037-mcp-stdio-transport-exclusion.md). + +### from_mcp + +```rust +pub struct FromMCP { + /// The MCP server's streamable HTTP endpoint URL. + endpoint: String, + /// Bearer token for the MCP server (from Capabilities at registration). + auth_token: Option, + /// The importing deployment's name for this MCP server (becomes the + /// operation namespace). + namespace: String, +} + +#[async_trait] +impl OperationAdapter for FromMCP { + async fn import(&self) -> Result, AdapterError>; +} +``` + +The adapter: + +1. Connects to the MCP server's streamable HTTP endpoint using rmcp's + `StreamableHttpClientTransport::from_uri(endpoint)` (the rmcp + `streamable_http.rs` client example shows the pattern: `client_info + .serve(transport).await`, then `client.list_tools()`, + `client.call_tool()`). On connection failure, returns + `AdapterError::DiscoveryFailed`; on 401, `AdapterError::Unauthorized`. +2. Calls `tools/list` → the list of MCP tools (name, description, + `inputSchema`, optional `outputSchema`). +3. For each tool, constructs a `HandlerRegistration`: + - `spec.name` = the tool name (or `namespace/tool_name` if a + namespace prefix is configured — same local-naming sugar as + `from_call`'s `FromCallConfig::namespace_prefix`, alkcall ADR-024 + §5). + - `spec.namespace` = the configured `namespace`. + - `spec.op_type` = `Mutation` (MCP tools are call/response; the MCP + spec doesn't have a native streaming/tool-subscription distinction + — `tools/call` returns a result. If MCP adds a streaming-tool + extension, a `Sub` mapping would be added.) All `from_mcp` + handlers are `HandlerKind::Once` (alkcall ADR-021); `from_mcp` + never produces a `StreamingHandler` (nor a `HandlerKind::Sink` — + the `Pub` type, alkcall ADR-046, has no MCP representation). + - `spec.visibility` = `Internal` (adapter-registered, alkcall + ADR-017). + - `spec.input_schema` = the tool's `inputSchema` (JSON Schema). + - `spec.output_schema` = depends on whether the tool declares + `outputSchema` (MCP 2025-06-18+): + - **`outputSchema` present** → `output_schema` = the declared + schema (converted from JSON Schema). The result arrives in + `CallToolResult.structured_content` and is composable with + local operations (the data matches the declared type). + - **`outputSchema` absent** (older MCP servers) → `output_schema` + = the MCP `ContentBlock` union (`text | image | audio | + resource | resource_link` — a well-defined MCP type, *not* + `Type.Unknown()`). The result arrives in + `CallToolResult.content` as a `Vec`. The common + sub-case is a single `Text` block (which older servers often + fill with JSON-stringified data), but the *type* is the + `ContentBlock` union regardless of what the text contains. + See "Output handling" below. + - `spec.error_schemas` = the MCP tool's error description mapped to + `ErrorDefinition` (alkcall ADR-016 — MCP tool definitions carry + error descriptions; the adapter maps them). + - `spec.access_control` = `AccessControl::default()`. + - `handler` = a forwarding handler (see Forwarding Handler below). + - `provenance` = `FromMCP`, `composition_authority: None`, + `scoped_env: None` (leaf — alkcall ADR-018). + - `capabilities` = the bearer token for the MCP server (injected by + the assembly layer at registration — see No-Env-Vars below). +4. Returns the bundles. The caller (the assembly layer) registers them + in the `OperationRegistry`. + +### Forwarding handler + +At call time, the `from_mcp` forwarding handler: + +1. Reads the call input (`serde_json::Value` — the tool arguments). +2. Calls `client.call_tool({ name: tool_name, arguments: input })` via + the rmcp client (the `streamable_http.rs` example shows + `client.call_tool(CallToolRequestParams::new(name).with_arguments(...))`). +3. On success: extracts the result from the `CallToolResult`, following + the `structuredContent`-preferred-over-content-blocks rule (see + "Output handling" below), wraps in a `ResponseEnvelope`, returns. +4. On `result.isError`: maps to a `CallError` with the MCP error content + (the TS `from_mcp.ts` handler shows the error mapping), returns. +5. The rmcp client connection is maintained for the lifetime of the + registration (the MCP server is a persistent streamable HTTP + endpoint, not a per-call connection). + +The handler is opaque to the `CallAdapter` — a `HandlerKind::Once` +wrapping an `Arc` that the registry dispatches. `alkcall` +never sees rmcp. + +### Output handling (structuredContent vs content blocks) + +MCP `CallToolResult` (rmcp `model.rs`) carries two result fields: +`content: Vec` (always present, defaults to `[]`) and +`structured_content: Option` (present when the tool declared +`outputSchema`). The `from_mcp` handler follows the same rule the TS +adapter (`@alkdev/operations/src/from_mcp.ts`) and the rmcp SDK +(`CallToolResult::into_typed`) use: + +- **`structured_content` present** (tool declared `outputSchema`): the + handler uses `structured_content` as the result, validated/cast + against the declared `output_schema`. This is the composable case — + the data matches the declared type, so a composing handler can use it + as a typed value. +- **`structured_content` absent** (older server, no `outputSchema`): + the handler maps `content: Vec` to the + `ContentBlock`-union `output_schema` (text/image/audio/resource/ + resource_link). The TS `mapMCPContentBlocks` shows the mapping; the + Rust `ContentBlock` enum (`rmcp/src/model/content.rs`) is the same + shape. The common sub-case is a single `Text` block — older servers + often JSON-stringify structured data into the `text` field. The + adapter does *not* attempt to `JSON.parse` the text heuristically + (fragile, not the adapter's concern); it carries the `ContentBlock` + union as the typed result. A consumer that knows the text is JSON can + parse it downstream. + +The `isError: true` case is handled separately (step 4 above) — it +maps to a `CallError`, not to the output handling path. + +### to_mcp + +```rust +pub fn to_mcp_service( + registry: Arc, + identity_provider: Arc, +) -> StreamableHttpService<...>; +``` + +`to_mcp` exposes the local registry's operations as a **fixed gateway +tool set** over streamable HTTP — not one MCP tool per operation. This +is the tool-gateway pattern (ADR-041): the LLM has a few tools in +context (search, schema, call, batch), not hundreds, and discovers +operations on demand through the gateway. See +[ADR-041](decisions/041-mcp-tool-gateway-pattern.md) for the +rationale (the tool-bloat problem, the `memory`/`worktree` tool pattern +that informed the design). + +The rmcp `simple_auth_streamhttp.rs` server example shows the +streamable-HTTP-service-into-axum-`Router` pattern: + +```rust +// From the rmcp example: +let mcp_service: StreamableHttpService = + StreamableHttpService::new( + || Ok(Counter::new()), + LocalSessionManager::default().into(), + StreamableHttpServerConfig::default(), + ); + +let protected_mcp_router = Router::new() + .nest_service("/mcp", mcp_service) + .layer(middleware::from_fn_with_state(token_store, auth_middleware)); +``` + +`alkhttp`'s `to_mcp` follows the same axum integration pattern, +but the rmcp `Service` impl is a gateway service (4 fixed tools) rather +than a per-operation tool registry. + +#### The gateway tool set + +`to_mcp` exposes four MCP tools that gate access to the full operation +registry: + +| MCP tool | Call protocol operation | Purpose | +|----------|------------------------|---------| +| `search` | `services/list` | List/search available operations (filtered by the caller's `AccessControl`). Returns names + descriptions, not full schemas. | +| `schema` | `services/schema` | Get an operation's full `OperationSpec` (input/output JSON Schemas, error schemas). | +| `call` | `call.requested` (Query/Mutation) | Invoke an operation by name with a JSON input. Returns the output or a typed error (alkcall ADR-016). | +| `batch` | multiple `call.requested` | Invoke multiple operations in one tool call (correlated request IDs, OQ-14). | + +The LLM calls `search` to discover operations, `schema` to learn an +operation's input shape, `call` to invoke. Same pattern as `man +` — discover on demand, don't preload. See ADR-041 for the +rationale. + +#### `Sub` and `Pub` exclusion + +The gateway exposes only `Query` and `Mutation` operations +(request/response). `Sub` operations (streaming responses — many +`call.responded` events) and `Pub` operations (streaming requests — +the consumer publishes chunks to a `HandlerKind::Sink` handler, +alkcall ADR-046) are both filtered out of `search` results and cannot +be invoked via `call` — MCP tool calls are request/response by +protocol design; neither a streaming response nor a client-side +publish fits the LLM tool-call pattern. This is unaffected by the +streaming handler work (alkcall ADR-021): the `StreamingHandler` type +and `invoke_streaming()` dispatch path exist in alkcall and are used +by `to_openapi`'s `/subscribe` endpoint (and `invoke_sink()` by +`/publish` — [ADR-068](decisions/068-gateway-publish-endpoint.md)), +but `to_mcp` does not expose them — it filters by `op_type` and only +dispatches `Query`/`Mutation` via `invoke()`. See +[ADR-041](decisions/041-mcp-tool-gateway-pattern.md) §2. + +#### `to_mcp` service behavior + +1. On MCP `tools/list`: returns the fixed gateway tool set (4 tools: + `search`, `schema`, `call`, `batch`), not the registry's + operations. The gateway tools have stable names and schemas; the + registry's operations are discovered through `search`. +2. On MCP `tools/call`: + - `search` → dispatches `services/list` (filtered by the caller's + `AccessControl`), returns operation names + descriptions. + - `schema` → dispatches `services/schema`, returns the + `OperationSpec`. + - `call` → dispatches `OperationRegistry::invoke()` (the same + dispatch spine the HTTP gateway uses — the gateway surface per + [ADR-047](decisions/047-remove-direct-call-http-surface.md), the + HTTP-to-call mapping per + [ADR-036](decisions/036-http-to-call-operation-mapping.md)). The + result is mapped to an MCP `CallToolResult` + (`structuredContent` for the output, or `isError: true` for a + `CallError` with typed `details` per alkcall ADR-016). + - `batch` → dispatches multiple `call.requested` events, returns + an array of results. +3. Auth: the Bearer middleware resolves the token via + `IdentityProvider::resolve_from_token()`, same as the HTTP server's + auth ([ADR-004](decisions/004-auth-as-shared-core.md)). The MCP + client authenticates by bearer token; no `PeerId` (browsers and MCP + clients are not alk peers — + [ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) + §4). `AccessControl` gates `search` results and `call` dispatch — the + LLM sees only what it's authorized to call. + +#### Shared dispatch spine with `to_openapi` + +`to_mcp`'s `call` tool and `to_openapi`'s `/call` endpoint share the +same dispatch spine: resolve caller identity (Bearer → +`IdentityProvider::resolve_from_token`) → build a root +`OperationContext` → `OperationRegistry::invoke()` → map the +`ResponseEnvelope` to the gateway's wire shape (`CallToolResult` for +MCP, HTTP JSON for OpenAPI). The wire framing, discovery listing +(`tools/list` vs `/search`), streaming (excluded in `to_mcp` vs +`to_openapi`'s `/subscribe` SSE and `/publish` NDJSON — the OpenAPI +gateway is 6 endpoints per +[ADR-068](decisions/068-gateway-publish-endpoint.md), while `to_mcp` +stays the 4 fixed tools), and server integration (rmcp +`StreamableHttpService` tower service vs axum route handlers) are +genuinely per-gateway and are not shared. See also +[http-adapters.md](http-adapters.md) §"Shared dispatch spine with +`to_mcp`". + +Research findings +(`/workspace/@alkdev/alknet/docs/research/alknet-http-gateway-factoring/findings.md`) +recommend extracting a **thin shared spine** (the concrete +`GatewayDispatch` struct holding `Arc` + +`Arc` with a `resolve + build_context + invoke` +method returning a `ResponseEnvelope`, named in alkcall ADR-021 and +extended with `invoke_streaming()` for the streaming path), **not** a +trait or gateway abstraction. The spine is small (~15–30 lines per +endpoint), but it is the one place where a divergence bug (identity +resolved differently, `OperationContext.internal` set inconsistently, +`CallError` mapped asymmetrically) would be a security/correctness +issue. The server-integration and wire-framing layers stay +per-gateway; a third gateway (GraphQL, gRPC) is not on the horizon, +and if one appears its server-integration layer needs its own shape +anyway. This is an implementation factoring note, not an ADR — the +decision is internal to `alkhttp` and does not cross crate boundaries. + +### No-Env-Vars + +The `from_mcp` forwarding handler reads the MCP server's bearer token +from `context.capabilities` (the same injection path as `from_openapi`), +not from `std::env::var`. The assembly layer injects the token at +registration; the handler reads it per-call. This is the no-env-vars +invariant ([ADR-014](decisions/014-secret-material-flow-and-capability-injection.md), +[overview.md](overview.md)). + +## Why + +MCP is the protocol editors and AI tools use to discover and call tools. +`from_mcp` lets the alk stack compose external MCP servers (a remote tool +server, a third-party MCP endpoint) into the call protocol — the same +composition pattern as `from_openapi` and `from_call`. `to_mcp` lets +external MCP clients (an editor, an AI tool) discover and call the local +deployment's operations through the MCP protocol, without those clients +needing to speak EventEnvelope. + +`to_mcp` uses the **tool-gateway pattern** (ADR-041): a fixed set of +meta-tools (`search`, `schema`, `call`, `batch`) gates access to the +full operation registry, so the LLM has a few tools in context instead +of hundreds. This addresses the tool-bloat problem — an LLM connecting +to a node with 200 operations gets 4 MCP tools, not 200, and discovers +operations on demand through `search` + `schema`. Same pattern as the +`memory` and `worktree` tools (one entry point, large dataset behind +it), and the same principle as Linux's `man` command (don't preload all +documentation; query on demand). + +The streamable-HTTP-only constraint (ADR-037) is a security position: +alkhttp does not import the MCP stdio RCE vector. The streamable HTTP +path is network-isolated, auth-gatable, and runs under alkhttp's +auth/identity/capabilities machinery — the same machinery that gates +every other HTTP request. + +## Constraints + +- **Streamable HTTP only.** Stdio is not built (ADR-037). The `mcp` + feature pulls in rmcp with streamable HTTP transport features only. +- **`from_mcp`-registered ops are `Internal` by default.** Composition + material, not directly callable from the wire (alkcall ADR-017). +- **`from_mcp` handlers read credentials from + `OperationContext.capabilities`.** No env vars + ([ADR-014](decisions/014-secret-material-flow-and-capability-injection.md)). +- **`to_mcp` is a pure projection.** Consumes the registry, does not + produce entries. Not an `OperationAdapter`. +- **MCP clients are not alk peers.** A browser or MCP client + connecting to `to_mcp` authenticates by bearer token, gets no + `PeerId`, is not in the peer graph + ([ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) + §4). +- **The `mcp` feature is optional.** A deployment that doesn't need MCP + doesn't compile rmcp. The default feature set is `h2` + `http1`. + +## Design Decisions + +| Decision | ADR | Summary | +|----------|-----|---------| +| MCP stdio transport excluded | [ADR-037](decisions/037-mcp-stdio-transport-exclusion.md) | Streamable HTTP only; stdio is not built | +| `to_mcp` tool-gateway pattern | [ADR-041](decisions/041-mcp-tool-gateway-pattern.md) | 4 fixed gateway tools (search/schema/call/batch), not one tool per operation; `Sub` and `Pub` excluded | +| `from_mcp` is an `OperationAdapter` | alkcall ADR-022 (Call Protocol Client and Adapter Contract) | Async trait (`alkcall::client`); produces `HandlerRegistration` bundles | +| `to_mcp` is a projection | alkcall ADR-022 (Call Protocol Client and Adapter Contract) | Consumes the registry, doesn't produce entries | +| Adapter-registered ops are `Internal` | alkcall ADR-017 (Privilege Model and Authority Context) | `from_mcp` ops are composition material | +| `from_mcp` provenance is a leaf | alkcall ADR-018 (Handler Registration, Provenance, and Composition Authority) | `composition_authority: None`, `scoped_env: None` | +| Error fidelity | alkcall ADR-016 (Operation Error Schemas) | MCP tool errors mapped to `ErrorDefinition`s | +| No-env-vars credential injection | [ADR-014](decisions/014-secret-material-flow-and-capability-injection.md) | Handler reads `context.capabilities`, not env vars | +| MCP clients are not alk peers | [ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) §4 | Bearer token, no `PeerId` | +| Streaming handler for subscriptions | alkcall ADR-021 (Streaming Handler for Subscription Operations) | `from_mcp` handlers are always `HandlerKind::Once` (MCP tools are request/response); `to_mcp` excludes `Sub` (and `Pub`) ops (unchanged by the streaming handler) | + +## Open Questions + +See [open-questions.md](open-questions.md) for full details. + +- **OQ-40** (resolved): reqwest client config — the shared + `ClientWithMiddleware` used by `from_mcp` (same client as + `from_openapi`). + +## References + +- [ADR-037](decisions/037-mcp-stdio-transport-exclusion.md) — the + stdio exclusion this document enforces +- [overview.md](overview.md) — adapter location map, feature gates +- the alkcall crate's `docs/architecture/client-and-adapters.md` — + `OperationAdapter` trait, `AdapterError` variants +- `/workspace/rust-sdk/` — MCP Rust SDK (rmcp v1.8.0); streamable HTTP + transport +- `/workspace/rust-sdk/crates/rmcp/src/model/tool.rs` — `Tool` with + `output_schema: Option>` (the `outputSchema` field) +- `/workspace/rust-sdk/crates/rmcp/src/model/content.rs` — `ContentBlock` + enum (text/image/audio/resource/resource_link — the fallback + `output_schema` type when `outputSchema` is absent) +- `/workspace/rust-sdk/crates/rmcp/src/model.rs` (~line 2868) — + `CallToolResult` with `content: Vec` and + `structured_content: Option` (the two result fields); see also + `into_typed` (~line 3057) for the SDK's own + structured-content-preferred-over-text-block fallback logic +- `/workspace/rust-sdk/examples/servers/src/simple_auth_streamhttp.rs` + — streamable HTTP MCP server with Bearer auth (the `to_mcp` pattern) +- `/workspace/rust-sdk/examples/clients/src/streamable_http.rs` — + streamable HTTP MCP client (the `from_mcp` pattern) +- `/workspace/@alkdev/operations/src/from_mcp.ts` — TypeScript prior art + (`createMCPClient`, `mapMCPContentBlocks`, the `MCPClientLoader`; the + `structuredContent`-preferred-over-content-blocks logic) +- `/workspace/@alkdev/operations/docs/architecture/adapters.md` — + TypeScript adapter architecture doc (the `from_mcp` `outputSchema`/ + `structuredContent` handling, the `MUTATION` tool-type decision) +- `/workspace/@alkdev/alknet/docs/research/alknet-http-gateway-factoring/findings.md` + — research on the shared dispatch spine between `to_mcp` and + `to_openapi` (recommendation: thin shared struct, not a trait) + +## Port notes + +Corrections applied during the alknet → alkhttp port, beyond mechanical +crate renames (`alknet-http` → `alkhttp`; `alknet-call`/`alknet-core` → +`alkcall`, with the adapter contract in `alkcall::client`, registry +types in `alkcall::registry`, core in `alkcall::core`): + +1. **`OperationType::Subscription` renamed `Sub`** (alkcall rename); + `OperationType::Pub` exists (producer→consumer streaming, + `HandlerKind::Sink`, wire event `call.published` — alkcall ADR-046). + The MCP gateway exclusion was a "Subscription excluded" claim in the + source; extended to cover both `Sub` (streaming responses) and `Pub` + (streaming requests) — MCP tool calls are request/response, so + neither fits. The section is retitled "`Sub` and `Pub` exclusion". + `to_mcp` stays 4 fixed tools. +2. **The OpenAPI gateway now has 6 endpoints.** `/publish` (Pub, NDJSON, + alkhttp ADR-068) joined the original five. The shared-dispatch-spine + cross-reference updated accordingly ("streaming (excluded in + `to_mcp` vs `to_openapi`'s `/subscribe` SSE and `/publish` + NDJSON)"; the `/subscribe` mention in the Sub-exclusion section now + cites `/publish` too). `to_mcp` has no `/publish` analog — the 4 + tool set is unchanged. +3. **ADR cross-reference remapping.** Call-protocol-internal decisions + now live in the alkcall crate and are cited textually (no relative + links across crates): old ADR-017 adapter contract → alkcall ADR-022 + (§5 mapping preserved); old ADR-022 handler registration → alkcall + ADR-018; old ADR-023 error schemas → alkcall ADR-016; old ADR-015 + privilege model → alkcall ADR-017; old ADR-049 streaming handler → + alkcall ADR-021 (the `GatewayDispatch` spine clause is alkcall ADR-021 + §7); Pub/`HandlerKind::Sink` → alkcall ADR-046. The old ADR-029 §5 + citation for `FromCallConfig::namespace_prefix` (peer-graph collision + rule) → alkcall ADR-024 §5 (the alkcall peer-graph routing model; + alkcall's ADR-029 is a different, later decision — aggregated peer-env + wiring). +4. **alkhttp-owned ADRs keep their numbers and are linked relatively:** + 014, 034, 004 (ported files exist); 037, 041 (being ported in + parallel — linked per instruction); 036, 047, 068 (linked by + inference — their ports are assumed, consistent with + http-adapters.md/http-server.md practice). The old ADR-036 "same + dispatch path the HTTP server uses" citation was extended with + ADR-047 (the gateway is the sole invoke path; ADR-047 supersedes + ADR-036's direct-call surface) so the reference reflects the current + architecture. +5. **Terminology.** "MCP clients are not alknet peers" → "not alk + peers"; "outside alknet" → "outside alkhttp"; "alknet's + auth/identity/capabilities machinery" → "alkhttp's". No call-protocol + server/client framing existed in the source; the remaining + server/client wording (rmcp streamable HTTP server/client, axum + server, remote MCP server, stdio-only MCP server) is MCP's inherent + transport directionality and is deliberately retained. +6. **Project naming.** "lets alknet compose external MCP servers" → + "lets the alk stack compose"; "calls alknet operations" → "the local + deployment's operations" in the Why prose. +7. **Links.** `../../decisions/` → `decisions/`; + `../../open-questions.md` → `open-questions.md` (sibling, pending + port); `../call/client-and-adapters.md` → textual reference to the + alkcall crate's `docs/architecture/client-and-adapters.md`; the + mono-repo research findings path is kept as a textual absolute path + (the findings doc was not ported). `overview.md` and + `http-adapters.md` are sibling links (overview.md pending port). + OQ-14/OQ-40 references kept as-is (OQ-14 is a call-protocol OQ — the + alkcall crate's `docs/architecture/open-questions.md`; OQ-40 is the + shared reqwest client config, resolved). \ No newline at end of file diff --git a/docs/architecture/http-server.md b/docs/architecture/http-server.md new file mode 100644 index 0000000..3e03bb7 --- /dev/null +++ b/docs/architecture/http-server.md @@ -0,0 +1,612 @@ +--- +status: draft +last_updated: 2026-08-27 +--- + +# HTTP Server + +The `HttpAdapter` — the `ProtocolHandler` for `h2` and `http/1.1` (and +WebSocket upgrade — see [websocket.md](websocket.md)). The `h3`/WebTransport +path is out of scope in alkhttp (ADR-069); this document covers how axum is +run over a bidirectional stream (BiStream), Bearer auth resolution, the +HTTP-to-call dispatch, the `/healthz` raw route, stealth decoy, and the +WebSocket upgrade route (which hands off to the channels session specified +in [websocket.md](websocket.md)). + +## What + +The `HttpAdapter` is constructed by the assembly layer with an +`Arc` (constructor injection, same pattern as +`SshAdapter` — see the alkcall crate's +`docs/architecture/client-and-adapters.md` for the adapter contract and +the alkcall crate's `docs/architecture/decisions/003-auth-as-shared-core.md` +(ADR-003) for the shared auth core) and an `Arc` (for +dispatching HTTP requests to call-protocol operations). It implements +`ProtocolHandler` for the standard HTTP ALPNs. + +```rust +pub struct HttpAdapter { + identity_provider: Arc, + registry: Arc, + /// The default handler for paths that are not registered operations + /// (stealth decoy). Configurable: a static site, a fake 404, a + /// redirect. Two-way-door default (ADR-010). + decoy: DecoyConfig, + /// Deployment-specific routes added by the assembly layer (ADR-046). + /// None = the default surface only. Custom routes are raw HTTP, not + /// call-protocol operations; they coexist with the default surface and + /// are not described by `to_openapi`. + extra_routes: Option, +} + +/// The stealth decoy surface for paths that are not registered +/// operations (and not `/healthz`, `/openapi.json`, the `to_openapi` +/// gateway endpoints `/search`/`/schema`/`/call`/`/batch`/`/subscribe`/ +/// `/publish`, or the MCP route). Set by the assembly layer at +/// `HttpAdapter` construction. The existence of the decoy path is fixed +/// by ADR-010; the variant is a two-way-door config default. +pub enum DecoyConfig { + /// Serve a fake `404 Not Found` (the default — matches the reference + /// implementation's "fake nginx 404"). + NotFound, + /// Serve a static site from a configured directory (the directory + /// path is the payload). For deployments that want a real decoy + /// website. + StaticSite { root: PathBuf }, + /// Redirect to a configured URL. + Redirect { to: String }, +} + +#[async_trait] +impl ProtocolHandler for HttpAdapter { + fn alpn(&self) -> &'static [u8]; // returns the configured ALPN + async fn handle(&self, connection: Connection, auth: &AuthContext) -> Result<(), HandlerError>; +} +``` + +The `HttpAdapter` registers for multiple ALPNs (`http/1.1`, `h2`). +The endpoint's `HandlerRegistry` maps each ALPN byte string to the same +adapter instance; `handle()` branches on `connection.remote_alpn()` to +pick the HTTP framing. For `http/1.1` and `h2`, the framing is hyper's +HTTP/1.1 or HTTP/2 over the bidirectional stream the connection yields. +WebSocket upgrade (see [websocket.md](websocket.md)) layers on top of the +same hyper connection driver — a WS upgrade is an HTTP/1.1 or HTTP/2 +request that switches protocols. There is no `h3` ALPN: h3/WebTransport +is out of scope in alkhttp (ADR-069). + +## Why + +HTTP is the standard external interface. Browsers, curl, axios, API +gateways, and load balancers all speak HTTP. Serving HTTP on the standard +ALPNs means any HTTP client can connect without knowing about the alk +stack — the TLS handshake negotiates `h2` or `http/1.1` normally. This is +the stealth mapping (ADR-010): the HTTP surface is the decoy for clients +that don't offer alk ALPNs, and the real external API surface for clients +that do know about the alk stack. + +## Architecture + +### Running axum over a bidirectional stream + +The `HttpAdapter::handle()` method for `h2`/`http/1.1`: + +1. Accepts one bidirectional stream from the connection + (`connection.accept_bi()` → `BiStream`). Over a multi-stream transport + this is one of many streams the connection provides; over a + single-stream connection (`Connection::from_bidi`, alkcall ADR-007 + "Connection::from_stream — Generic Single-Stream Connections") it is + the one stream, yielded once then `ConnectionClosed`. Either way, + `accept_bi` returns the joined `BiStream` (`AsyncRead + AsyncWrite`, + per alkcall ADR-005 "BiStream Type Definition") the adapter needs — + the handler code is transport-agnostic. +2. Wraps the `BiStream` as a hyper `TokioIo`-compatible stream — the + same byte stream hyper expects for an HTTP connection. +3. Constructs the axum `Router` (built once at adapter construction, + cloned per connection — axum `Router` is `Clone` and cheap to clone). +4. Hands the stream + the axum router to hyper's connection driver + (`hyper::server::conn::http1::Builder` or + `http2::Builder::serve_connection`), which reads HTTP frames, parses + them, dispatches to axum routes, and writes HTTP responses. +5. Returns when the HTTP connection closes (the client disconnects or + the stream ends). + +The axum `Router` is built once at adapter construction with the +`Arc` and `Arc` embedded in its +state; cloning the `Router` per connection clones the `Arc`s (cheap, +shared state), so every request handler has access to the registry and +identity provider through the router's state. + +The axum `Router` is the single routing surface for HTTP requests. It +contains: + +- **The `to_openapi` gateway endpoints** (`/search`, `/schema`, `/call`, + `/batch`, `/subscribe`, `/publish` — ADR-042, with `/publish` per + alkhttp ADR-068). These 6 fixed endpoints are the sole invoke path over + HTTP: an HTTP client invokes an operation via `POST /call` with + `{ "operation": "/{service}/{op}", "input": {...} }`, discovers + available operations via `GET /search` + (`AccessControl`-filtered), and learns an operation's shape via `GET + /schema`. `POST /subscribe` is the SSE streaming invoke path (body + `{ operation, input }`, same shape as `/call`, response + `text/event-stream`). `POST /publish` is the Pub streaming invoke path + (producer→consumer streaming via `call.published`; alkcall ADR-046 + "Publish Operation Type and `HandlerKind::Sink`"). There is no + per-operation `POST /{service}/{op}` direct-call surface — the + gateway is the invoke path (ADR-047 supersedes ADR-036's direct-call + surface; the simplified contract is a few fixed endpoints, not a + per-operation REST tree). `/call` and `/subscribe` dispatch through + `OperationRegistry::invoke()`; `/publish` dispatches through + `OperationRegistry::invoke_sink()`; `/search` and `/schema` dispatch + the `services/list` / `services/schema` discovery ops. +- `GET /healthz` (raw route, no auth, no call protocol). +- `GET /openapi.json` (serves the `to_openapi` projection — the OpenAPI + document that *describes* the 6 gateway endpoints. The doc describes + the 6 fixed endpoints, and the per-caller operation surface is + discovered via `/search`, not preloaded into `paths`. The doc carries + `info.version` (semver) tracking the gateway endpoint contract — + consumers detect breaking changes via the major version (ADR-045)). +- The stealth decoy fallback (unknown paths). +- (Feature-gated) `POST /mcp` (the `to_mcp` streamable HTTP service — + [http-mcp.md](http-mcp.md)). +- **Deployment-specific custom routes** (ADR-046). The assembly layer + may inject an `axum::Router` of extra routes at `HttpAdapter` + construction — e.g., an OpenAI-compatible proxy at + `/v1/chat/completions` that dispatches into the registry. These are + raw HTTP, not call-protocol operations: not in the + `OperationRegistry`, not discoverable via `/search`, not described + by `to_openapi`. The default surface's reserved paths take precedence + on collision; custom routes namespace away from the reserved set + naturally (`/v1/...`). A deployment that passes no extra routes gets + exactly the default surface above. A deployment that wants a + REST-like per-operation HTTP surface (the former direct-call shape) + builds it as a custom route projection (ADR-047 §4). See ADR-046 and + §"Custom routes" below. + +A single HTTP/2 or HTTP/1.1 connection multiplexes multiple requests +over the one bidirectional stream (HTTP/2 multiplexing is native; +HTTP/1.1 is sequential). The axum router handles each request on a +tokio task; the hyper driver manages the connection lifetime. + +### HTTP-to-call dispatch (the gateway's `/call`; ADR-042, ADR-047) + +An HTTP client invokes an operation via the gateway's `/call` endpoint: + +1. The axum route handler for `POST /call` reads the JSON body + `{ "operation": "/fs/readFile", "input": {...} }`. +2. It resolves the caller's identity from the `Authorization: Bearer` + header via `identity_provider.resolve_from_token(&AuthToken { raw: + token_bytes })`. +3. It constructs the root `OperationContext` (caller identity, the + registration bundle's capabilities, the connection's env composition) + and dispatches through the `OperationRegistry::invoke()` — the same + dispatch path the `CallAdapter` uses for `alk/call` wire requests. +4. The response (`ResponseEnvelope`) is serialized as the HTTP response + body (JSON). Errors map to HTTP status codes (see Error Mapping + below). + +`Internal` operations (ADR-015) return `404` (`NOT_FOUND`) — the gateway +dispatches only `External` operations, and the caller discovers which +`External` operations it can call via the `AccessControl`-filtered +`/search` endpoint. This is the per-caller API surface property that +the direct-call surface (removed, ADR-047) lacked: an HTTP client cannot +stub its toe on a path for an operation it can't call, because there is +no per-operation path — `/search` tells it what it can call, `/call` +invokes it, and the `AccessControl` check runs on `/call` regardless. + +`/batch` follows the same dispatch path with an array of +`{ operation, input }` pairs (OQ-14); `/subscribe` follows it with the +SSE streaming projection (below); `/publish` follows it with the +`invoke_sink()` dispatch and `HandlerKind::Sink` handlers (alkcall +ADR-046), projecting the HTTP request body as the initiator's stream +into the operation's sink. + +### Streaming projection (SSE — the gateway's `/subscribe`) + +A `Sub` operation invoked via the gateway's `POST /subscribe` endpoint +projects its `call.responded` stream as Server-Sent Events. The request +body is `{ operation, input }` (the same flat JSON shape as `/call`); +the response is `text/event-stream` (negotiated via +`Accept: text/event-stream` on the `POST`). The axum route handler: + +- Sets `Content-Type: text/event-stream`. +- Calls `GatewayDispatch::invoke_streaming()` (alkcall ADR-021 + "Streaming Handler for Subscription Operations") — the streaming + analogue of `invoke()`, returning a `BoxStream`. The + security invariants are identical to `invoke()`: `internal: false`, + `forwarded_for: None`, same capabilities, same `scoped_env`, same ACL + check before dispatch. The two methods diverge only on the return shape + (stream vs single envelope). +- For each `ResponseEnvelope` the stream yields, writes an SSE `data:` frame: + `Ok(value)` → `data:` frame with the output serialized as JSON; `Err` → + SSE error event with the `CallError` serialized, then close (an `Err` is + terminal — the stream ends after it, matching the wire protocol's + `call.error` semantics). +- On natural stream end (the streaming handler's stream completes), closes + the SSE stream (normal end — corresponds to `call.completed` on the wire). +- On `call.aborted` or HTTP client disconnect (detected as the response + writer closing), drops the stream future — `Drop` guards release the + handler's resources, and the abort cascade runs per alkcall ADR-020 + "Abort Cascade for Nested Calls". + +This is the HTTP/1.1 + HTTP/2 streaming projection. Over WebSocket +([websocket.md](websocket.md)), the subscription projects directly onto +the channels session — `call.responded` events as binary messages on the +subscribed channel, no SSE framing. WebTransport (`h3`) is out of scope +in alkhttp (ADR-069). + +**The streaming dispatch path.** Pre-ADR-021, `subscribe_handler` called +`GatewayDispatch::invoke()` (single response) and wrapped the one +`ResponseEnvelope` in a one-event SSE stream — a placeholder that couldn't +stream a real `Sub` op. alkcall ADR-021 adds `GatewayDispatch:: +invoke_streaming()` and the underlying `OperationRegistry:: +invoke_streaming()`, giving `/subscribe` a real streaming dispatch path +to call. See alkcall ADR-021 and [http-adapters.md](http-adapters.md) for +the `from_openapi` SSE forwarding handler that feeds streaming handlers +from external `text/event-stream` responses. + +### One-directional projection (HTTP request/response) + +The HTTP/1.1 + HTTP/2 surface is a **lossy, one-directional projection** +of the call protocol. HTTP is request/response: the consumer initiates, +the producer responds. The call protocol is bidirectional — both sides +can initiate calls (see the alkcall crate's +`docs/architecture/call-protocol.md` §"Bidirectional Calls": the accept +side can call operations on the connect side just as the connect side +calls operations on the accept side). The HTTP projection carries only +the consumer→producer call direction; the producer→consumer call +direction has no HTTP expression (there is no HTTP mechanism for the +producer to initiate a request to the consumer). `Sub` streaming is the +one partial exception — the producer streams `call.responded` frames +back over the SSE response — but even there, the *call* is +consumer-initiated; only the *results* flow producer→consumer. + +This is a structural property of HTTP, not a design choice in this +crate. **WebSocket restores the bidirectional call model for browsers** +(see [websocket.md](websocket.md)): a WS connection is a long-lived +full-duplex channel over which either side can send `call.requested` +frames in either direction — the call protocol's native bidirectionality +applies unchanged (alkcall ADR-015 "Call Protocol Stream Model" — +stream-agnostic correlation; a WS message stream is another +`BiStream`-satisfying transport). WebTransport (`h3`) would restore it +via native multi-stream multiplexing, but WebTransport is out of scope +in alkhttp (ADR-069) — WebSocket is the v1 browser bidirectional path. +The HTTP/1.1 + HTTP/2 surface is the projection for clients that only +speak HTTP; WebSocket is the surface for browser clients that speak the +call protocol in both directions. + +### WebSocket browser path (ADR-048, ADR-067) + +A browser (or any WS client) upgrades an HTTP/1.1 or HTTP/2 request to +WebSocket (RFC 6455) at the upgrade path `/alk/channels`; the resulting +full-duplex WS connection carries the **channels protocol** (alkhttp +ADR-067; framing per alkcall ADR-034 "Channels Wire Format — 8-Byte +Chunk Header"): 8-byte chunk multiplexing, with channel 0 +pre-negotiated as `alk/call` (alkcall ADR-036 "Channel 0 Is +Pre-Negotiated `alk/call`"). The shared `Dispatcher` +runs on channel 0 — the WS path is a channels session whose channel 0 is +the call-protocol session, and it is the surface that **restores the call +protocol's native bidirectionality for browsers** (unlike the +one-directional HTTP projection above). The WS path carries the **native +session, not the HTTP gateway shape** (ADR-048): the gateway endpoints +are HTTP-only, discovery is via `services/list`/`services/schema` as +call-protocol ops, and subscriptions project as native `call.responded` +events (no SSE). + +The full WS handler specification — the upgrade route, the channels +framing, the dispatch handoff to the shared `Dispatcher` on channel 0, +bidirectionality, the connection-local Layer 2 overlay, the "browsers are +not peers" rationale (ADR-034 §4; the old ADR-044 §5 amendment is out of +scope with WebTransport per ADR-069), the streaming +projection, and the deferred `from_wss` adapter — is at +[websocket.md](websocket.md). `h3`/WebTransport is out of scope in +alkhttp (ADR-069); the ALPN-stream-proxy path (alknet ADR-040) is an +alknet concern and is not available here. + +### Auth + +Inbound HTTP auth is `Authorization: Bearer `, resolved via +`IdentityProvider::resolve_from_token()` (the auth handler table: +`HttpAdapter`, Bearer header, `resolve_from_token`). Bearer-only is the +auth mechanism for the default surface; other HTTP auth schemes (Basic, +API key in query param) are not implemented and would be added as axum +middleware (two-way door). This is recorded in +[ADR-036](decisions/036-http-to-call-operation-mapping.md) §Auth; +the resolution mechanism (`resolve_from_token`) is from alkcall ADR-003 +"Auth as Shared Core (IdentityProvider)", and the connection-level +observability (`set_identity`) is OQ-11 (resolved). + +- Bearer-only is the auth mechanism. Basic auth, API keys in query + params, and other HTTP auth schemes are not implemented. A deployment + that needs a different auth scheme adds it as axum middleware + (two-way door), but the default surface is Bearer-only. +- The `HttpAdapter` constructor-injects `Arc`, + same pattern as `SshAdapter`. +- An unauthenticated request to an operation with `AccessControl` + restrictions returns `401` (no token) or `403` (token present but + insufficient scopes). The call protocol's `FORBIDDEN` protocol code + maps to `403`; `NOT_FOUND` (Internal op) maps to `404`. +- The HTTP handler stores the resolved identity on the `Connection` for + observability (`connection.set_identity(identity)`), same as the call + protocol handler. + +### Error Mapping + +Call-protocol `CallError` codes (alkcall ADR-016 "Operation Error +Schemas") map to HTTP status codes: + +| Call `code` | HTTP status | Notes | +|-------------|-------------|-------| +| `NOT_FOUND` (operation not registered, or Internal op) | `404` | | +| `FORBIDDEN` (insufficient scopes, or unauthenticated) | `401` (no token) / `403` (token present) | | +| `INVALID_INPUT` (schema mismatch) | `422` | | +| `TIMEOUT` | `504` | `retryable: true` | +| `INTERNAL` | `500` | | +| Operation-level domain code with `http_status` (alkcall ADR-016) | the declared `http_status` | `from_openapi`-imported ops carry the original status | +| Operation-level domain code without `http_status` | `500` | | + +The `retryable` field from `CallError` maps to an HTTP `Retry-After` +hint for `503`/`429`-class errors. The mapping is a two-way-door +default (the exact status for ambiguous codes can be refined +additively); the one-way constraint is that protocol-level and +operation-level codes are distinct (alkcall ADR-016) and +`from_openapi`-imported codes are prefixed `HTTP_` to avoid +collision with protocol codes. + +### `/healthz` (raw route) + +`GET /healthz` is a raw HTTP route outside the call protocol — no auth, +no operation registration, no `OperationContext`. It returns `200 OK` +with a plain-text body (e.g., `"ok"`) if the endpoint is healthy. This +is the infrastructure endpoint load balancers and orchestrators call; +it must work before identity is resolvable. + +Other operational endpoints (metrics, dashboard) are call-protocol +operations if built (`/metrics/list`, `/dashboard/view`), not raw HTTP +routes. `healthz` is the one exception. See ADR-036. + +### Stealth decoy + +For paths that are not the gateway endpoints (`/search`, `/schema`, +`/call`, `/batch`, `/subscribe`, `/publish`), `/healthz`, +`/openapi.json`, the MCP route, or a custom route per ADR-046), the HTTP +handler serves a decoy. The decoy is configurable (`DecoyConfig`): + +- A fake `404 Not Found` (the default — matches the reference + implementation's "fake nginx 404"). +- A static site (served from a configured directory). +- A redirect (to a configured URL). + +The decoy is the stealth surface: a port scanner or a client that +doesn't offer alk ALPNs connects on `h2`/`http/1.1` and sees the decoy. +Real services use `alk/ssh`, `alk/call`, etc. The decoy config is a +two-way-door default (an operator picks what to serve); the *existence* +of the stealth path is fixed by ADR-010. Custom routes (ADR-046) take +precedence over the decoy — a path matched by a custom route is served +by it, not the decoy; the decoy is the fallback for paths matched by +neither the default surface nor a custom route. + +### Custom routes (ADR-046) + +A deployment that needs HTTP endpoints outside the default surface +(gateway + `/healthz` + `/openapi.json` + MCP) injects +them as an `axum::Router` at `HttpAdapter` construction. The classic use +case: an OpenAI-compatible proxy at `/v1/chat/completions` that wraps a +call-protocol operation (the deployment parses the OAI request, invokes +an `openai/chat` or `agent/chat` op via `OperationRegistry::invoke()`, +reformats the response as an OAI response). The hub is a standard alk +node *plus* a deployment-specific HTTP surface. + +Custom routes: + +- Are **raw HTTP**, not call-protocol operations — not registered in the + `OperationRegistry`, not discoverable via `/search`, not in the + `to_openapi` gateway doc. +- **May** dispatch into the registry via + `OperationRegistry::invoke()` with a proper `OperationContext` + (caller identity from the resolved bearer token) — the OAI proxy + does this. Or they may be pure HTTP (a webhook receiver, a static + asset server) that never touches the registry. +- Run under the **default Bearer-auth middleware**; a route that wants + different auth applies its own axum middleware (the deployment owns + its custom routes' middleware stack). +- **Do not collide** with the reserved default-surface paths + (`/search`, `/schema`, `/call`, `/batch`, + `/subscribe`, `/publish`, `/healthz`, `/openapi.json`, the MCP + route) — the default surface wins on collision; custom routes + namespace away naturally (`/v1/...`). (ADR-047 removed the direct-call + `POST /{service}/{op}` surface, so `/{service}/{op}` is no longer a + reserved path; a deployment that builds a per-operation projection as + a custom route is the one case where `/{service}/{op}` patterns + appear, subject to the same collision rule.) +- Are **not versioned** by `to_openapi` (ADR-045 versions the gateway + contract, not custom routes). The deployment versions its own custom + routes however it wants. +- Are **immutable after construction** (matches OQ-04 / ADR-010's + static-registration constraint; the `HttpAdapter` router is built once + at startup). + +The extension point is additive: a deployment that passes `None` gets +exactly the default surface. The mechanism (the constructor parameter) +is the one-way door — once downstream deployments build against it, it's +a contract (ADR-046). The specific routes a deployment adds are a +two-way door (add/remove freely). See +[ADR-046](decisions/046-assembly-layer-custom-http-routes.md). + +## Constraints + +- **The gateway is the sole invoke path over HTTP (ADR-042, ADR-047).** + The 6 gateway endpoints (`/search`, `/schema`, `/call`, `/batch`, + `/subscribe`, `/publish`) are the only way to invoke operations over + HTTP. There is no per-operation `POST /{service}/{op}` direct-call + surface — the simplified contract is a few fixed endpoints, not a + per-operation REST tree. A client invokes an operation via `POST /call` + with `{ "operation": "/{service}/{op}", "input": {...} }`; it discovers + what it can call via the `AccessControl`-filtered `/search`. The + per-caller API surface is the default (the Gitea failure mode — every + operation gets a path, every caller sees the full surface — is + structurally impossible). A deployment that wants a REST-like + per-operation HTTP surface builds it as a custom route projection + (ADR-046, ADR-047 §4). +- **`External` operations only.** `Internal` operations return `404` + on the gateway's `/call`, matching the call protocol's `NOT_FOUND`. +- **Bearer-only auth.** `Authorization: Bearer` → + `resolve_from_token`. Other HTTP auth schemes are not implemented. +- **No secret material in HTTP responses.** The call protocol carries no + secret material (alkcall ADR-010 "Secret Material Flow and Capability + Injection"); the HTTP handler inherits this constraint. Capabilities + are used for outbound calls (`from_openapi`), never serialized into + HTTP response bodies. +- **`/healthz` is raw.** No auth, no call protocol. The one raw route. +- **WebSocket is the browser bidirectional path (ADR-048, ADR-067).** A + browser + upgrades an HTTP request to WS at `/alk/channels` and the connection + carries the channels protocol (8-byte chunk multiplexing; alkcall + ADR-034) — channel 0 is pre-negotiated as `alk/call` (alkcall ADR-036) + and runs the shared `Dispatcher`; the **native session, not the + gateway shape** (the gateway endpoints are HTTP-only; discovery via + `services/list`/`services/schema` as call-protocol ops). + `h3`/WebTransport is out of scope in alkhttp (ADR-069); the + ALPN-stream-proxy (alknet ADR-040) is not available. The `h3` ALPN + and its feature gate are not implemented. Full WS handler spec: + [websocket.md](websocket.md). +- **Custom routes are raw HTTP, not call-protocol operations + (ADR-046).** The assembly layer injects an `axum::Router` of extra + routes at `HttpAdapter` construction. They are not in the + `OperationRegistry`, not discoverable via `/search`, not in the + `to_openapi` doc. They may dispatch into the registry via + `OperationRegistry::invoke()` (the OAI-compatible proxy pattern) or + be pure HTTP. The default surface's reserved paths take precedence on + collision. A deployment that passes no extra routes gets the default + surface unchanged. + +## Design Decisions + +| Decision | ADR | Summary | +|----------|-----|---------| +| ~~Direct path mapping (HTTP path = operation path)~~ | ~~[ADR-036](decisions/036-http-to-call-operation-mapping.md)~~ | **Superseded by ADR-047** — direct-call surface removed; gateway `/call` is the sole invoke path | +| Gateway is the sole invoke path over HTTP | [ADR-042](decisions/042-openapi-gateway-pattern.md), [ADR-047](decisions/047-remove-direct-call-http-surface.md) | 6 fixed gateway endpoints (`/search`/`/schema`/`/call`/`/batch`/`/subscribe`/`/publish`); `POST /call` with `{ operation, input }` is the invoke path; per-caller `AccessControl`-filtered `/search` is the discovery; no per-operation HTTP paths | +| `/publish` gateway endpoint for Pub operations | alkhttp ADR-068 | `POST /publish` projects an HTTP request body as the initiator's stream into `invoke_sink()` / `HandlerKind::Sink` (alkcall ADR-046) | +| `to_openapi` published-spec versioning | [ADR-045](decisions/045-to-openapi-gateway-spec-versioning.md) | `/openapi.json` carries `info.version` (semver) tracking the gateway contract, not the operation set | +| SSE projection for subscriptions (`/subscribe`) | [ADR-036](decisions/036-http-to-call-operation-mapping.md) §Streaming, [ADR-042](decisions/042-openapi-gateway-pattern.md) §2 | `call.responded` stream → SSE frames; the gateway's `/subscribe` endpoint is the entry point | +| `/healthz` is a raw route | [ADR-036](decisions/036-http-to-call-operation-mapping.md) | No auth, no call protocol | +| Stealth decoy | [ADR-010](decisions/010-alpn-router-and-endpoint.md) | HTTP handler on standard ALPNs serves decoy for non-gateway, non-custom, non-`/healthz` paths | +| Bearer auth via `resolve_from_token` | alkcall ADR-003 "Auth as Shared Core (IdentityProvider)" | HTTP handler credential source (settled) | +| WebSocket is the browser bidirectional path | [ADR-048](decisions/048-websocket-native-session-not-gateway.md), alkhttp ADR-067 | Browsers upgrade to WS at `/alk/channels`; the channels protocol over binary messages (8-byte chunk multiplexing, alkcall ADR-034), channel 0 pre-negotiated as `alk/call` (alkcall ADR-036); `h3`/WebTransport out of scope (ADR-069). WS carries the native session, not the gateway shape (gateway endpoints are HTTP-only). Full spec: [websocket.md](websocket.md) | +| Browsers are not alk peers | [ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) §4 | Bearer token, no `PeerId`, connection-local overlay (addressability vs. bidirectionality) — full rationale in [websocket.md](websocket.md) | +| Error mapping (call codes → HTTP status) | alkcall ADR-016 "Operation Error Schemas" | Protocol/operation codes distinct; `HTTP_` prefix for imported | +| Custom HTTP routes from the assembly layer | [ADR-046](decisions/046-assembly-layer-custom-http-routes.md) | `extra_routes: Option` at construction; raw HTTP, not operations; default surface takes precedence on collision | +| Streaming handler for subscriptions (`invoke_streaming()`) | alkcall ADR-021 "Streaming Handler for Subscription Operations" | `GatewayDispatch::invoke_streaming()` returns `BoxStream`; `/subscribe` pipes it to SSE; replaces the one-event placeholder with the real streaming dispatch path | + +## Open Questions + +See the alkhttp crate's `docs/architecture/open-questions.md` (when +present) and the alkcall crate's `docs/architecture/open-questions.md` +for full details. + +- **OQ-39** (resolved): `to_openapi` published-spec versioning — + resolved by [ADR-045](decisions/045-to-openapi-gateway-spec-versioning.md): + `info.version` semver tracks the gateway endpoint contract (not the + operation set); the per-caller operation surface is discovered via + `/search` and does not bump the version. +- **OQ-40** (resolved): reqwest client config and connection pooling — + `ClientWithMiddleware` + middleware stack; the outbound HTTP client + used by `from_openapi`/`from_mcp`. + +## References + +- [ADR-036](decisions/036-http-to-call-operation-mapping.md) — the + HTTP-to-call mapping this server implements +- [ADR-048](decisions/048-websocket-native-session-not-gateway.md) + — WS carries the native session (via the channels protocol, channel 0 + pre-negotiated as `alk/call`), not the HTTP gateway shape; the gateway + endpoints are HTTP-only. +- alkhttp ADR-068 — the `/publish` gateway endpoint (Pub operations, + `invoke_sink()` dispatch) +- alkhttp ADR-069 — h3/WebTransport out of scope in alkhttp +- alkcall ADR-003 "Auth as Shared Core (IdentityProvider)" — Bearer → + `resolve_from_token` +- alkcall ADR-014 "irpc Was Never Integrated — Hand-Rolled EventEnvelope + Framing" — the `EventEnvelope` wire format +- alkcall ADR-015 "Call Protocol Stream Model" — the `Dispatcher` + (stream-agnostic; runs over the channels session unchanged) +- alkcall ADR-034 "Channels Wire Format — 8-Byte Chunk Header" — the + channels framing the WS path carries +- alkcall ADR-036 "Channel 0 Is Pre-Negotiated `alk/call`" — channel 0 + as the call-protocol session +- [websocket.md](websocket.md) — the full WS browser path spec (upgrade + route, channels framing, dispatch on channel 0, bidirectionality, + connection-local overlay, streaming projection, the deferred + `from_wss` adapter) +- [overview.md](overview.md) — crate overview, adapter location map +- [http-adapters.md](http-adapters.md) — `from_openapi`/`to_openapi` +- the alkcall crate's `docs/architecture/call-protocol.md` — + `EventEnvelope` wire format, `Dispatcher`, bidirectional calls +- the alkcall crate's `docs/architecture/client-and-adapters.md` — the + adapter contract (`CallAdapter` dispatch path), `IdentityProvider` + constructor injection +- the alkcall crate's `docs/architecture/operation-registry.md` — + `OperationRegistry::invoke()` / `invoke_streaming()` / `invoke_sink()`, + the dispatch path HTTP requests hit + +## Port notes + +Corrections applied while porting from +`alknet/docs/architecture/crates/http/http-server.md` (temporary +scaffolding for review; remove once the port is accepted): + +- Crate names: `alknet-http` → alkhttp; `alknet-core`/`alknet-call` + types → alkcall (`alkcall::core::auth`, `alkcall::core::types`, + `alkcall::registry`, `alkcall::protocol`, adapter contract in + `alkcall::client`). The alkcall dependency is `alkcall` 0.1.1 + (old alknet-core and alknet-call merged into it). +- Transport: "QUIC bidirectional stream" / `(SendStream, RecvStream)` + phrasing replaced with the joined `BiStream` (`AsyncRead + + AsyncWrite`) returned by `Connection::accept_bi()`; handlers wrap it + with `TokioIo`. Single-stream vs multi-stream notes kept (true in + alkcall: `Connection::from_bidi` yield-once, alkcall ADR-007). +- Terminology: producer/consumer for call-protocol roles (both sides + can initiate); "server"/"client" kept only for HTTP's inherent + directionality. "alknet" node/peer references → "alk" node/alk stack. +- OperationType: `Subscription` → `Sub` (renamed in alkcall). Added + `OperationType::Pub` / `HandlerKind::Sink` coverage: `/publish` + endpoint (alkhttp ADR-068), `invoke_sink()` dispatch, and alkcall + ADR-046 citations. Gateway count updated 5 → 6 everywhere (reserved + paths, DecoyConfig doc comment, constraints, design table). +- WebSocket: the WS path now carries the **channels protocol** (8-byte + chunk multiplexing, alkcall ADR-034), channel 0 pre-negotiated as + `alk/call` (alkcall ADR-036) with the shared `Dispatcher` on channel + 0 — not bare `EventEnvelope` binary messages. Upgrade path + `/alknet/call` → `/alk/channels`. Bidirectionality and + connection-local overlay statements retained. The `EventEnvelope` + framing is still the channel-0 payload (alkcall ADR-014, amended by + alkcall ADR-035). Cited alkhttp ADR-067 for the WS-carries-channels + decision. +- WebTransport/h3: removed from scope — "deferred per ADR-044" / + webtransport.md references replaced with "out of scope in alkhttp + (ADR-069)"; webtransport.md references deleted. The old ADR-034 §4 / + ADR-044 §5 amendment chain on the "browsers are not peers" rationale + is trimmed to ADR-034 §4 (the ADR-044 amendment covered the + WebTransport revival framing, which is out of scope per ADR-069). + The old ADR-040 ALPN-stream-proxy is + cited as an alknet concern (not ported to alkhttp). +- Links: `../../decisions/` → `decisions/`; links into the old + mono-repo (`../call/...`, `../core/...`) converted to textual + references to "the alkcall crate's docs/architecture/.md". + ADR cross-refs to call-internal decisions re-mapped to alkcall ADRs: + ADR-012→alkcall ADR-015 (stream model), ADR-016→alkcall ADR-020 + (abort cascade), ADR-022→alkcall ADR-018 (handler registration), + ADR-023→alkcall ADR-016 (error schemas), ADR-024→alkcall ADR-019 + (registry layering), ADR-049→alkcall ADR-021 (streaming handler), + ADR-004→alkcall ADR-003 (auth shared core), ADR-007→alkcall ADR-005 + (BiStream type), call wire format→alkcall ADR-014, Pub/Sink→alkcall + ADR-046. alkhttp-local ADRs (010, 034, 036, 042, 045, 046, 047, 048) + keep their numbers as `decisions/NNN-.md` links. +- References section: removed the `/workspace/@alkdev/pubsub` TypeScript + prior-art file paths (alknet-research artifact; the alkcall ADR-014 + prior-art note retains the lineage) and the old mono-repo + `../core/auth.md`, `../core/endpoint.md`, `../call/...`, + `open-questions.md` links (open-questions.md not yet ported to + alkhttp; referenced textually). The ALPN prefix is `alk/` (alkcall + ADR-004 amendment 1 shortened `alknet/` → `alk/`), so `alknet/ssh`, + `alknet/call` → `alk/ssh`, `alk/call`, and the stealth-decoy "doesn't + offer alknet ALPNs" phrasing → "alk ALPNs". \ No newline at end of file diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md new file mode 100644 index 0000000..e092b7b --- /dev/null +++ b/docs/architecture/open-questions.md @@ -0,0 +1,89 @@ +# Open Questions + +Centralized tracker. Format follows the SDD process +(`docs/sdd_process.md` §Open Questions Format). Ported alknet OQs that +resolved before the extraction are recorded in the resolved section +with their resolutions; new alkhttp OQs start at OQ-01. + +## Status legend + +`open` | `resolved` | `deferred(scope)` | `partially resolved` + +## Open + +### OQ-01: WS ↔ byte-stream adaptation semantics + +- **Origin**: [websocket.md](websocket.md), [ADR-067](decisions/067-websocket-carries-channels.md) +- **Status**: open +- **Priority**: high +- **Question**: The channels demux consumes bytes (`read_exact` on the + 8-byte header + payload); the mux writes chunks as contiguous byte + sequences. A WebSocket is message-oriented. The adapter's contract + needs nailing down before implementation: + (a) inbound buffer bound (bounded channel between the WS read task + and the `AsyncRead` half — what bound, what policy on overflow); + (b) write-side chunk completeness (the adapter assumes the mux emits + each chunk as one contiguous `write_all` — verify against alkcall's + `MuxRunner` and codify, or add an internal chunking layer); + (c) flush mapping (`AsyncWrite::flush` → WS message emission point); + (d) close mapping (WS close code → transport EOF → REQ-CH-02 + teardown; and does `AsyncWrite::shutdown` map to a WS Close frame or + to a zero-length chunk sentinel?). +- **Blocked on**: nothing (implementation-blocking, not + decision-blocking — resolve during implementation of the WS adapter) + +### OQ-02: `/publish` body framing details + +- **Origin**: [ADR-068](decisions/068-gateway-publish-endpoint.md) +- **Status**: open +- **Priority**: medium +- **Resolution**: (pending) +- **Question**: The exact first-line convention for naming the target + operation (first line `{operation, chunk}` vs `?operation=` query + parameter vs required header), and where a terminal error envelope + lives (final NDJSON line of a JSON error object vs plain HTTP status + with JSON body). Must settle before the gateway contract's + `info.version` bumps ([ADR-045](decisions/045-to-openapi-gateway-spec-versioning.md)). + +### OQ-03: `from_wss` reconnection semantics + +- **Origin**: [ADR-070](decisions/070-from-wss-consumer-adapter.md) +- **Status**: open +- **Priority**: medium +- **Resolution**: (pending) +- **Question**: On a dropped WSS connection, does `from_wss` + auto-reconnect + re-discover (`services/list`) + reconcile imported + registrations, or does v1 surface call failures (`INTERNAL`, + retryable) and leave the policy to the assembly layer? The v1 + default (fail with retryable errors) is documented in ADR-070; the + question is whether a built-in policy is ever warranted. + +### OQ-04: Browser client library ownership + +- **Origin**: [websocket.md](websocket.md) +- **Status**: open +- **Priority**: low +- **Blocked on**: a concrete browser consumer (the alk UI) needing the + JS/TS client for channels-over-WS. The framing contract is the + alkcall BAST document (`chunk-header.bast.json`); the client library + lives outside this crate. Deferred(scope: browser UI work). + +## Resolved (ported) + +Resolutions inherited from the alknet architecture; recorded here for +reference. Full rationale in the alknet mono-repo's open-questions.md +and the cited ADRs. + +| OQ (alknet) | Title | Resolution | Where it lives now | +|-------------|-------|------------|--------------------| +| OQ-11 | Handler-level auth resolution observability | Resolved: resolved identity stored on `Connection` via `set_identity` | alkcall core; [http-server.md](http-server.md) §Auth | +| OQ-13 | Operation path format | Resolved: `/{service}/{op}` is the operation path format (gateway bodies carry it; no direct-call surface) | [ADR-036](decisions/036-http-to-call-operation-mapping.md), [ADR-047](decisions/047-remove-direct-call-http-surface.md) | +| OQ-17 | Call protocol client and adapter contract | Resolved: `OperationAdapter` async trait; `to_*` projections | alkcall ADR-022; [ADR-017](decisions/017-call-protocol-client-and-adapter-contract.md) | +| OQ-24 / OQ-26 | Operation error schemas / `AdapterError` variants | Resolved: protocol/operation codes distinct; `HTTP_` prefix; `AdapterError` `#[non_exhaustive]` | alkcall ADR-016; [ADR-023](decisions/023-operation-error-schemas.md) | +| OQ-37 | X.509 outgoing-only / three peer roles | Resolved: browsers are not peers | [ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) | +| OQ-39 | `to_openapi` published-spec versioning | Resolved: `info.version` semver tracks the gateway endpoint contract | [ADR-045](decisions/045-to-openapi-gateway-spec-versioning.md) | +| OQ-40 | reqwest client config and connection pooling | Resolved: `ClientWithMiddleware` + retry + Retry-After middleware; rebuild-and-swap | [http-adapters.md](http-adapters.md) §HTTP client | +| OQ-12 | TLS identity provisioning | Resolved (alknet): browsers require X.509; provisioning itself is an alknet concern | [ADR-027](decisions/027-tls-identity-redesign-acme-rawkey-decoupling.md); [ADR-069](decisions/069-webtransport-out-of-scope.md) | + +Not carried: OQ-38 (WebTransport standalone relay scope) — moot here; +the relay is an alknet concern ([ADR-069](decisions/069-webtransport-out-of-scope.md)). \ No newline at end of file diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md new file mode 100644 index 0000000..be51445 --- /dev/null +++ b/docs/architecture/overview.md @@ -0,0 +1,303 @@ +--- +status: draft +last_updated: 2026-08-27 +--- + +# alkhttp — Overview + +The HTTP interface crate: serves inbound HTTP on standard ALPNs (with +WebSocket upgrade carrying the channels protocol for browser +bidirectional access) and hosts the HTTP-backed call-protocol adapters. +This document covers the crate's two roles, its dependency edges, and +the adapter location map. Component details are in the sibling +documents. + +## What + +alkhttp is the HTTP protocol handler crate for the alk stack. It serves +two roles in one crate ([ADR-039](decisions/039-http-server-and-client-host-colocated.md)): + +1. **HTTP server** — a `ProtocolHandler` (`HttpAdapter`) that accepts + HTTP/2 and HTTP/1.1 connections on the standard IANA ALPNs (`h2`, + `http/1.1`), plus WebSocket upgrade (for browser bidirectional + access to the call protocol). It serves REST APIs, the + `to_openapi`/`to_mcp` projections of local call-protocol operations, + the `/healthz` operational endpoint, and the decoy surface for + stealth mode. The WS path carries the **channels protocol** + ([ADR-067](decisions/067-websocket-carries-channels.md)): the 8-byte + chunk header multiplexes N channels over the WS connection; channel 0 + is pre-negotiated as `alk/call` and runs the shared `Dispatcher`. +2. **HTTP client host** — the home of the HTTP-transport-backed call + adapters: `from_openapi` (import external HTTP APIs described by an + OpenAPI document, using `reqwest` for outbound calls), `from_mcp` + (import remote MCP tools over streamable HTTP, using `reqwest`), + `from_jsonschema` (import a single non-OpenAPI HTTP endpoint), and + `from_wss` (import a remote alk node's operations over a WSS + channels connection — same-protocol importer). The reverse + projections `to_openapi` (generate an OpenAPI doc from the local + registry's `External` operations) and `to_mcp` (expose local ops as + MCP tools over streamable HTTP, using `axum`) also live here. + +Both directions share the same HTTP dependencies (`axum` for serving, +`reqwest` for calling out), which is why they live in one crate rather +than being split into a server crate and a client crate. + +## Why + +The crate's purpose is to be the HTTP interface library for downstream +crates that need to expose an HTTP interface. A downstream consumer (a +hub deployment, a browser-facing service, the alknet assembly layer) +wires `HttpAdapter` into the handler registry for the standard HTTP +ALPNs and gets a full HTTP surface: the gateway projection of the call +protocol, OpenAPI discovery, MCP tool exposure, and WebSocket for +browser bidirectional access to both the call protocol and data +channels. + +### The producer/consumer model + +The call protocol is bidirectional — both sides can initiate calls — +and the channels protocol is symmetric — either side can open +channels. **Producer** and **consumer** therefore describe *what a side +does on a given operation or channel*, not what it is as a connection +endpoint: + +- A **producer** (call) registers operations on an `OperationRegistry` + and runs a `Dispatcher`; a **consumer** (call) uses a `CallConnection` + to invoke them. On one connection both sides can be producers of some + ops and consumers of others. +- A **producer** (channels) accepts data channels (registers openable + ALPNs); a **consumer** (channels) opens them via the per-ALPN open + ops on channel 0. Both sides can open and accept on the same + connection. + +The HTTP surface maps onto this model as follows: + +- The `HttpAdapter` (gateway + WS upgrade) is the **producer-facing + surface**: it runs the shared `Dispatcher` on channel 0 for WS + sessions, and the gateway dispatches into the local registry for HTTP + requests. A browser or HTTP client is a **consumer** of those ops. +- The WS session is fully bidirectional: the browser side can also be a + producer (registering ops via the connection-local Layer 2 overlay, + opening channels), making the hub a consumer of browser ops on the + same connection. +- The `from_*` adapters are **consumer-side**: they import external + surfaces (OpenAPI, MCP, WSS) into the local registry as ops the local + process can then consume or compose. +- The `to_*` adapters are **producer-side projections**: they expose + the local registry's ops through foreign protocols (OpenAPI, MCP). + +Connection direction (who opened the HTTP/TCP connection) is +independent of these roles, exactly as in the call protocol. + +A note on the "from/to" direction model: the `from_openapi`/`to_openapi` +and `from_mcp`/`to_mcp` adapters are *inherently directional* because +OpenAPI and MCP are client/server protocols — one side serves, the +other calls. That directionality is a property of those protocols, not +of the call protocol itself. The call protocol is bidirectional. The +HTTP/1.1 + HTTP/2 surface inherits HTTP's request/response constraint +and projects the call protocol one-directionally (consumer→producer +calls only). **WebSocket restores the call protocol's native +bidirectionality for browsers** — and now carries the channels protocol +besides it. + +## Dependencies + +``` +alkhttp +├── alkcall (call + channels protocol, vendored core types, +│ registry, adapter contract — the sole alk dep) +├── axum (HTTP server — Router, extractors, middleware, WebSocket upgrade) +├── hyper / hyper-util(hTTP/1.1 + HTTP/2 framing; axum is built on hyper) +├── reqwest (HTTP client — from_openapi/from_mcp forwarding) +│ + reqwest-middleware, reqwest-retry +├── tokio-tungstenite (WSS client — from_wss; behind the `wss` feature) +├── yaml_serde (YAML parse for from_openapi YAML input — ADR-051) +├── openapiv3 (OpenAPI doc model for from_openapi/to_openapi) +└── rmcp (MCP streamable HTTP — feature-gated behind `mcp`) +``` + +> **Note:** the `h3`/WebTransport dependency is **not** in the +> dependency tree and is not planned — WebTransport is out of scope in +> alkhttp ([ADR-069](decisions/069-webtransport-out-of-scope.md)); when +> a browser needs WebTransport it will be provided by the alknet layer, +> which owns transports. The browser bidirectional path is WebSocket +> (native axum support, no new dependency). + +### The alkcall dependency ([ADR-003](decisions/003-crate-decomposition.md) Amendment 1) + +alkhttp depends on alkcall. The old rule "no handler crate depends on +another handler crate" survives as "no HTTP adapter depends on another +transport adapter": alkcall is the protocol-foundation crate (it owns +the vendored core types — `Connection`, `ProtocolHandler`, `BiStream`, +`AuthContext`, `IdentityProvider`, `Capabilities`, `OwnershipProvider`, +`HandlerError`, `StreamError` — plus the registry, dispatch, and wire +format). alkhttp depending on alkcall is "HTTP uses the call protocol +types," not "HTTP depends on another transport." + +alkcall stays transport-agnostic — no `reqwest`, no `axum`, no HTTP +dependencies. The `from_openapi`/`from_mcp`/`from_jsonschema`/`from_wss` +forwarding handlers are opaque `Arc`-shaped registrations +from the registry's perspective: constructed by alkhttp at registration +time, stored in `HandlerRegistration`, dispatched by the `Dispatcher` +which doesn't know `reqwest` or `tokio-tungstenite` is involved. + +## ALPNs + +| ALPN | Handler | Transport | Browser? | +|------|---------|-----------|----------| +| `http/1.1` | `HttpAdapter` | HTTP/1.1 over a `BiStream` (+ WS upgrade) | Yes (WS upgrade for bidirectional) | +| `h2` | `HttpAdapter` | HTTP/2 over a `BiStream` (+ WS upgrade) | Yes (WS upgrade for bidirectional) | +| `h3` | — | — | Out of scope ([ADR-069](decisions/069-webtransport-out-of-scope.md)); an alknet concern | + +These are standard IANA ALPN strings, not `alk/`-prefixed. Any HTTP +client connects without knowing about the alk stack — the TLS handshake +negotiates `h2` or `http/1.1` normally, and the `HttpAdapter` serves +HTTP. This is the stealth mapping +([ADR-010](decisions/010-alpn-router-and-endpoint.md)). + +The `HttpAdapter` registers for `http/1.1` and `h2`. The endpoint's +handler registry maps each ALPN to the same adapter instance; the +handler branches on `connection.remote_alpn()` to pick the framing. +WebSocket upgrade rides on either HTTP version (RFC 6455 over HTTP/1.1; +extended CONNECT over HTTP/2). + +## Adapter Location Map + +The decomposition principle: the adapter trait lives where the types +live (alkcall); the adapter implementations live where their transport +dependencies live (alkhttp). + +``` +alkcall (transport-agnostic — no HTTP client, no HTTP server) +├── OperationAdapter trait (the contract — async) +├── from_call (same-protocol importer over a call connection) +├── CallConnection / CallClient (outbound call surface) +└── ChannelsAdapter / ChannelClient (channels protocol, both sides) + +alkhttp (owns HTTP server + HTTP client + WSS client) +├── HttpAdapter (axum server — inbound HTTP on h2/http1.1 + WS upgrade route) +├── [WS upgrade → channels session] (not an adapter — hands the WS byte stream to the +│ channels machinery; see websocket.md, ADR-067) +├── from_openapi (parse OpenAPI doc + reqwest forwarding handler) +├── from_jsonschema (single-endpoint reqwest forwarding handler — ADR-066) +├── to_openapi (generate OpenAPI doc from local registry — the gateway projection) +├── from_mcp (feature-gated) (import remote MCP tools over streamable HTTP — reqwest) +├── to_mcp (feature-gated) (expose local ops as MCP tools over streamable HTTP — axum) +└── from_wss (feature-gated) (import a remote node's ops over WSS — ADR-070) +``` + +alkcall never sees the HTTP client. The forwarding handlers are opaque +from the registry's perspective. alkcall stays lean; alkhttp owns all +HTTP directions and the WSS consumer. + +## Feature Gates + +```toml +[features] +default = ["h2", "http1"] # the HTTP surface (incl. WebSocket upgrade for browsers) +mcp = ["dep:rmcp"] # from_mcp / to_mcp (streamable HTTP only — ADR-037) +wss = ["dep:tokio-tungstenite"] # from_wss consumer adapter (ADR-070) +``` + +- `h2` + `http1` (default): the `axum` + `hyper` HTTP/1.1 + HTTP/2 + server, including WebSocket upgrade for browser bidirectional access. + This is the surface all clients — including browsers, via WS upgrade — + use. +- `mcp`: the `rmcp` dependency with streamable HTTP transport features + only. Adds `from_mcp`/`to_mcp`. See [http-mcp.md](http-mcp.md) and + [ADR-037](decisions/037-mcp-stdio-transport-exclusion.md). +- `wss`: the `tokio-tungstenite` dependency. Adds `from_wss`. See + [ADR-070](decisions/070-from-wss-consumer-adapter.md). + +**`yaml_serde` is not feature-gated** — YAML OpenAPI is a first-class +input format (some providers publish YAML-only schemas), not an edge +case ([ADR-051](decisions/051-yaml-input-for-from-openapi.md) §3). + +## The No-Env-Vars Invariant + +The `from_openapi`/`from_mcp`/`from_jsonschema`/`from_wss` forwarding +handlers are the **credential injection point** for the no-env-vars +architecture. The path: + +``` +vault (alkvault) → assembly layer → Capabilities + → HandlerRegistration.capabilities → OperationContext.capabilities + → handler reads context.capabilities.get("") + → injects into outbound HTTP Authorization header (or WSS auth) +``` + +This makes any downstream `std::env::var("OPENAI_API_KEY")` read +unreachable — the assembly layer never calls `Default::default()` on a +provider; it constructs them with vault-derived credentials, or routes +outbound calls through adapter operations that carry the credential in +`Capabilities`. + +**This is a spec-level invariant**: no handler reads outbound +credentials from any source other than `OperationContext.capabilities`. +All adapter implementations in alkhttp are verified against this +invariant. See +[ADR-014](decisions/014-secret-material-flow-and-capability-injection.md). + +## Architecture (component pointers) + +- **[http-server.md](http-server.md)** — the `HttpAdapter` for `h2`/ + `http/1.1` (+ the WS upgrade route): how axum is run over a + `BiStream`, Bearer auth resolution, the `/healthz` raw route, stealth + decoy, the 6-endpoint gateway dispatch, and the WS upgrade route + (which hands off to the channels session). +- **[websocket.md](websocket.md)** — the WebSocket browser bidirectional + path: the channels protocol over WS, framing via the WS↔byte-stream + adapter, channel 0 and the shared `Dispatcher`, bidirectionality, + connection-local Layer 2 overlay, the browsers-are-not-peers + rationale, streaming (native, no SSE). +- **[http-adapters.md](http-adapters.md)** — `from_openapi`, + `from_jsonschema`, `to_openapi`, and `from_wss`. Error fidelity per + [ADR-023](decisions/023-operation-error-schemas.md). +- **[http-mcp.md](http-mcp.md)** — `from_mcp`/`to_mcp` (feature-gated), + streamable HTTP only (ADR-037), the rmcp integration. + +## Design Decisions + +| Decision | ADR | Summary | +|----------|-----|---------| +| HTTP-to-call operation mapping | [ADR-036](decisions/036-http-to-call-operation-mapping.md) | ~~Direct path mapping~~ — **routing superseded by ADR-047**; non-routing clauses survive (SSE, auth, `/healthz`, stealth, error mapping) | +| MCP stdio transport exclusion | [ADR-037](decisions/037-mcp-stdio-transport-exclusion.md) | Streamable HTTP only; stdio is not built (RCE vector) | +| WebSocket carries the channels protocol | [ADR-067](decisions/067-websocket-carries-channels.md) | WS = in-line channels substrate; channel 0 = `alk/call`; upgrade path `/alk/channels`; new in alkhttp | +| Gateway `/publish` endpoint | [ADR-068](decisions/068-gateway-publish-endpoint.md) | 6th gateway endpoint for `Pub` ops; NDJSON request body → `call.published` chunks; new in alkhttp | +| WebTransport out of scope | [ADR-069](decisions/069-webtransport-out-of-scope.md) | h3/WebTransport removed from alkhttp scope; an alknet concern; supersedes ADR-044's deferral framing | +| `from_wss` consumer adapter | [ADR-070](decisions/070-from-wss-consumer-adapter.md) | Same-protocol importer over WSS; `wss` feature; new in alkhttp | +| WS carries the native session, not the gateway shape | [ADR-048](decisions/048-websocket-native-session-not-gateway.md) | Amended by ADR-067: channel 0 carries the native call session; gateway endpoints are HTTP-only | +| HTTP server + client host colocated | [ADR-039](decisions/039-http-server-and-client-host-colocated.md) | One crate for server + adapters (shared HTTP deps, shared mapping) | +| `to_mcp` tool-gateway pattern | [ADR-041](decisions/041-mcp-tool-gateway-pattern.md) | 4 fixed gateway tools (search/schema/call/batch); Sub and Pub excluded | +| `to_openapi` gateway pattern | [ADR-042](decisions/042-openapi-gateway-pattern.md), [ADR-047](decisions/047-remove-direct-call-http-surface.md) | Fixed gateway endpoints are the sole HTTP invoke path (no per-operation `POST /{service}/{op}`); per-caller AccessControl-filtered `/search` is the discovery; extended with `/publish` (ADR-068) | +| Assembly-layer custom HTTP routes | [ADR-046](decisions/046-assembly-layer-custom-http-routes.md) | `extra_routes: Option` at construction; default surface takes precedence on collision | +| alkhttp is protocol-foundation-dependent | [ADR-003](decisions/003-crate-decomposition.md) Am. 1 | alkhttp depends on alkcall alone (types, not a peer transport) | +| Bearer auth via `resolve_from_token` | [ADR-004](decisions/004-auth-as-shared-core.md) | HTTP handler credential source + resolution (settled; alkcall ADR-003 owns the mechanism) | +| Stealth mode = HTTP handler on standard ALPNs | [ADR-010](decisions/010-alpn-router-and-endpoint.md) | Decoy for unknown paths (settled) | +| Adapter-registered ops are `Internal` | [ADR-015](decisions/015-privilege-model-and-authority-context.md) | `from_openapi`/`from_mcp`/`from_jsonschema`/`from_wss` produce `Internal` leaves (record: alkcall ADR-017) | +| `OperationAdapter` trait is async | [ADR-017](decisions/017-call-protocol-client-and-adapter-contract.md) | HTTP adapters implement the async trait (record: alkcall ADR-022) | +| `to_*` adapters are projections | [ADR-017](decisions/017-call-protocol-client-and-adapter-contract.md) | `to_openapi`/`to_mcp` consume the registry, don't produce entries (record: alkcall ADR-022) | +| Error schema fidelity | [ADR-023](decisions/023-operation-error-schemas.md) | `from_openapi` maps HTTP status → `HTTP_` codes; `to_openapi` projects back (record: alkcall ADR-016) | +| Browsers require X.509 | [ADR-027](decisions/027-tls-identity-redesign-acme-rawkey-decoupling.md) | Browser-facing TLS uses X.509 (TLS provisioning itself is an alknet concern) | +| Browsers are not alk peers | [ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) §4 | Browser over WS = bearer token, no `PeerId` | + +## Open Questions + +See [open-questions.md](open-questions.md) for full details. + +- **OQ-01** (open): WS message ↔ byte-stream adaptation — the channels + demux consumes bytes; axum WS yields messages. The adapter's + buffering/flush semantics are the core implementation risk. +- **OQ-02** (open): `/publish` body framing details — NDJSON chunk + shape, error position. +- **OQ-03** (open): `from_wss` reconnection semantics. + +## References + +- The alkcall crate — `docs/architecture/` for the call and channels + protocol decisions; `README.md` §"Roles and Composition" for the + producer/consumer role definitions. +- The alkvault crate — secrets, feeding `Capabilities` at the assembly + layer. +- The alknet mono-repo — endpoint/transport concerns and the source + specs this crate was ported from. \ No newline at end of file diff --git a/docs/architecture/websocket.md b/docs/architecture/websocket.md new file mode 100644 index 0000000..77f13e1 --- /dev/null +++ b/docs/architecture/websocket.md @@ -0,0 +1,377 @@ +--- +status: draft +last_updated: 2026-08-27 +--- + +# WebSocket — the Browser Bidirectional Path (Channels over WS) + +WebSocket is the browser bidirectional path to the call protocol and +the channels protocol. A WS session carries the **channels protocol** +([ADR-067](decisions/067-websocket-carries-channels.md)): the 8-byte +chunk header multiplexes N logical channels over the WS binary message +stream; **channel 0 is pre-negotiated as `alk/call`** and carries the +native call-protocol session, dispatched by the shared `Dispatcher`. +This supersedes the alknet design where WS carried one bare +`EventEnvelope` per message. + +A WS session is a **native channels session, not the HTTP gateway +shape** — the gateway endpoints (`/search`, `/schema`, `/call`, +`/batch`, `/subscribe`, `/publish`) are the HTTP one-directional +projection and **do not appear on the WebSocket path** +([ADR-048](decisions/048-websocket-native-session-not-gateway.md), as +amended by ADR-067). Discovery is via `services/list`/`services/schema` +as call-protocol operations on channel 0. + +## What + +The WebSocket path is an axum WS upgrade handler on the same +`HttpAdapter` that serves `h2`/`http/1.1` +([http-server.md](http-server.md)). A browser (or any WS client — Node, +a native app, the `from_wss` consumer +([ADR-070](decisions/070-from-wss-consumer-adapter.md)) in reverse) +opens an HTTP/1.1 or HTTP/2 request to the upgrade path, authenticates +by bearer token on the upgrade request, and the resulting full-duplex +WS connection is a channels connection: the 8-byte chunk format +(alkcall ADR-034) runs over it exactly as over TCP+TLS. + +## Why + +WebSocket is the HTTP-family transport that restores the alk protocols' +native bidirectionality for browsers — HTTP/1.1 + HTTP/2 are +request/response (a one-directional projection of the call protocol), +while WS is a full-duplex, long-lived, framed-message channel. Carrying +the **channels protocol** (rather than a bare envelope stream, the +alknet choice) makes a browser session structurally identical to a Rust +in-line channels session (TCP+TLS): one session shape, one dispatch +path, data channels included. The decision is +[ADR-067](decisions/067-websocket-carries-channels.md); the decision +that the WS path carries the native session rather than the gateway +shape is [ADR-048](decisions/048-websocket-native-session-not-gateway.md). + +### The consumer-side mirror: `from_wss` + +The same session serves the outbound direction: a Rust process dials a +remote node's WSS endpoint and consumes its operations +([ADR-070](decisions/070-from-wss-consumer-adapter.md)). Server upgrade +and consumer dial share the WS↔byte-stream adapter; the session halves +are alkcall's (`ChannelsAdapter` accept path vs `ChannelClient` +consumer path). + +## Architecture + +### The WS upgrade handler + +The WS upgrade is an HTTP/1.1 or HTTP/2 request handled by an axum +route on `HttpAdapter`'s router. The handler: + +1. Receives the HTTP upgrade request (axum's `WebSocketUpgrade` + extractor). +2. Resolves the caller's identity from the `Authorization: Bearer` + header via `identity_provider.resolve_from_token(&AuthToken { raw: + token_bytes })` (`AuthToken` is in `alkcall::core::auth`) — the same + auth path as any HTTP request + ([http-server.md](http-server.md) §"Auth"). The upgrade is rejected + (`401`) if no token is present; insufficient scopes for any op the + browser later calls surface as `FORBIDDEN` at call time, not at + upgrade time (the upgrade doesn't know which ops the browser will + call). +3. Upgrades to WebSocket (axum's `WebSocketUpgrade::on_upgrade`), + producing a full-duplex `WebSocket` stream. +4. Adapts the WS message stream to a byte stream (see §"Framing" below + — the adapter is this spec's core new piece). +5. Wraps the byte stream as a `Connection` (`Connection::from_bidi`, + ALPN `alk/channels`). +6. Runs the channels accept path: the in-line demux loop + (alkcall `ChannelsAdapter`) installs channel 0 via the + `install_channel_zero` hook — constructing channel 0's + `CallConnection` (identity attached) and running + `Dispatcher::run_loop_single_stream` on it. +7. Data channels (1..N) route per the deployment's openable-ALPN + registrations; the browser opens them via the per-ALPN open ops on + channel 0 (alkcall ADR-047), the same mechanism any consumer uses. + +The **default upgrade path is `/alk/channels`** (was `/alknet/call` in +the alknet design). The path must not collide with the reserved +gateway/`/healthz`/`/openapi.json`/MCP/custom-route paths per +[ADR-046](decisions/046-assembly-layer-custom-http-routes.md)'s +collision rule; `/alk/channels` namespaces away from the reserved set +naturally. The upgrade runs over HTTP/1.1 (RFC 6455) or HTTP/2 +(extended CONNECT, RFC 8441); axum/hyper supports both, and the handler +does not branch on which — the WS frame stream is the same once the +upgrade completes. + +### Framing: WS messages carry chunks; channel 0 carries length-prefixed envelopes + +The wire layering: + +``` +WS binary message (message boundary = transport frame) +└── chunk header [channel_id: u32 BE][length: u32 BE] (8 bytes) + └── payload + channel 0: [len: u32 BE][EventEnvelope JSON] (call frame) + channel N: handler-owned framing (opaque to channels) +``` + +- **One WS binary message = one chunk** (header + payload). The WS + message boundary is the chunk boundary — no re-splitting, no + coalescing across messages required. +- **Channel 0's payload is the call protocol's frame format** + (alkcall ADR-014): a 4-byte big-endian length prefix + UTF-8 JSON + `EventEnvelope`. This is exactly the framing channel 0 uses over + TCP+TLS; the WS path is not special at this layer. +- **Data-channel payloads are opaque** — the handler owns its framing + (alkcall ADR-035: no `stream_type` anywhere in the channels layer). +- **Text WS messages are rejected** with a protocol-level close (code + `1002`). All frames are binary. +- The alknet design's "one envelope = one WS message, no length + prefix" rule is superseded; see + [ADR-048](decisions/048-websocket-native-session-not-gateway.md)'s + status amendment. + +#### The WS ↔ byte-stream adapter + +The channels machinery reads and writes **bytes** (`AsyncRead` + +`AsyncWrite`): the demux does `read_exact` on the 8-byte header and +payload; the mux writes chunk bytes. A WebSocket is +**message-oriented**: axum yields whole `Message`s, and writes are +whole `Message`s. The adapter bridges the two, in both directions: + +- **Inbound (WS → bytes):** a background task pops WS binary messages + and appends their bytes into a shared buffer; the `AsyncRead` half + drains the buffer. Backpressure: the reader task awaits a bounded + buffer slot before admitting the next message (bound: OQ-01). +- **Outbound (bytes → WS):** the `AsyncWrite` half accumulates bytes + into a pending buffer and emits exactly one WS binary message per + chunk — the mux writes header+payload as one contiguous + `write_all` (the channels mux's `MuxRunner` composes chunks + atomically), so the adapter's write-side job is buffer-until-chunk- + complete, then flush as one message. Flush semantics and the + chunk-completeness assumption are OQ-01 items to verify against + alkcall's `MuxRunner` behavior. +- **Close mapping:** WS close (either side) → transport EOF → the + demux clears all channels (REQ-CH-02: every handler sees EOF) and + channel 0's dispatch loop fails outstanding pendings with + `connection closed`. + +The adapter is shared with the `from_wss` consumer path +([ADR-070](decisions/070-from-wss-consumer-adapter.md)) — one +implementation, both directions. + +### Dispatch: channel 0 = the shared `Dispatcher`, unchanged + +Channel 0's session is the alknet design's native session, verbatim: + +- For `call.requested`: runs `AccessControl::check(identity)` against + the op's `AccessControl`, dispatches via the registry if allowed, + returns `FORBIDDEN` (→ `call.error`) before the handler runs if not. +- For `call.responded`/`call.completed`/`call.aborted`: correlates by + `id` via the pending map (keyed by request ID, not by transport — + alkcall ADR-015). +- For `call.published` (initiator-side `Pub` ops): routed to the + matching sink (alkcall ADR-046). +- Writes response frames back through channel 0's shared writer. + +Peer authorization flows through `AccessControl::check` against the +resolved identity — an op with `AccessControl::default()` is callable +by any authenticated browser; an op with `required_scopes` only by +identities whose scopes satisfy them; an op with +`Visibility::Internal` is never callable from the wire (`NOT_FOUND` +before ACL). This is alkcall ADR-017's model; see also +[ADR-015](decisions/015-privilege-model-and-authority-context.md). + +### Data channels for browsers + +A browser opens a data channel exactly as any channels consumer: + +1. Calls the per-ALPN open op on channel 0 + (`channels//sub` or `channels//pub` — alkcall ADR-047). +2. The producer side (the hub) checks the open against the registered + openable ALPNs and `check_open(identity)`, allocates the + `channel_id`, and returns it. +3. Chunks with that `channel_id` flow over WS messages; both sides + reassemble the `BiStream`. + +The deployment decides which ALPNs are openable — the browser reaches +exactly what a Rust consumer on an in-line connection would reach. +This is what makes the WASM-SSH-client-in-a-browser use case (the one +that motivated the alknet WebTransport track) workable over WS: the +SSH byte stream rides a data channel. + +### Bidirectionality + +The WS channels session inherits both protocols' native +bidirectionality (alkcall ADR-015 for calls; the channels protocol is +symmetric by construction): + +- **Calls:** both sides can send `call.requested` on channel 0. The + browser calls hub ops; the hub can call browser-registered ops over + the same session, same pending map, same framing. +- **Channels:** the browser can open data channels (via open ops); the + hub can open them toward the browser if the browser side registers + openable ALPNs (a browser that is also a producer). + +The common case — a browser that registers no ops and opens no +channels — is a use-case scoping, not an architectural limitation. + +### Connection-local overlay + +A browser over WS has no `PeerId` on the hub's side. Any ops the +browser registers land in a **connection-local Layer 2 overlay** +(alkcall ADR-019; the mechanism the hub reaches via the live +connection handle's `overlay_env()`). When the WS connection closes +(browser closes the tab, network drops), the overlay and all its +registered ops drop — no explicit deregistration. The hub reaches +browser ops through the connection handle, not through +`PeerRef::Specific` (the browser is not a peer — see below). + +### Streaming: native `call.responded` events, no SSE + +A `Sub` operation invoked on channel 0 streams `call.responded` +frames as channel-0 chunks — **no SSE framing**. SSE is the h2/http1.1 +streaming projection (the gateway's `/subscribe` endpoint per +[ADR-042](decisions/042-openapi-gateway-pattern.md)); on WS it is +unnecessary because WS is already a framed full-duplex channel. The +browser receives `call.responded` events one per frame, with the same +`id` correlating them to the original `call.requested`; +`call.completed` closes the subscription; `call.aborted` closes it +with an error frame. This is identical to how subscriptions work over +any in-line channels transport. + +A `Pub` operation (browser as initiator) publishes `call.published` +frames from the browser into channel 0 — the HTTP gateway's `/publish` +([ADR-068](decisions/068-gateway-publish-endpoint.md)) has no WS +equivalent because the native mechanism already exists. + +On WS client disconnect (the browser closes the tab mid-subscription), +the session teardown detects the close and aborts in-flight +subscriptions — the abort cascade runs per alkcall ADR-020. + +### Browsers are not alk peers + +A browser over WS authenticates by bearer token, gets no `PeerId`, +does not enter `PeerCompositeEnv`, and its registered ops (if any) land +in the connection-local overlay. The rationale +([ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) §4, +amended by alknet ADR-044 §5) is a load-bearing distinction: + +**"Peer" means an addressable node in the call-protocol peer graph** — +a stable `PeerId`, reachable via `PeerRef::Specific`, whose identity is +stable across reconnects. It does *not* mean "any endpoint that +exchanges calls during a live session." A browser is the second thing +but not the first: + +1. **No stable cryptographic identity of its own.** A peer entry is + anchored to fingerprints the *peer* presents and the local node + pins. A browser presents a bearer token the *hub* issued — hub + bookkeeping, not pinnable identity. There is nothing to put in + fingerprints. +2. **Ephemeral.** Close the tab → connection dies → the + connection-local overlay dies with it. A peer entry keyed to a + browser would be permanently dead within seconds. +3. **Not addressable from other nodes.** `PeerRef::Specific` resolves + through a peer entry; another node has no way to reach "the browser + currently connected to hub-A." The hub holds that connection as a + live handle, not a peer-graph entry. + +The browser is a **bidirectional call target during a live session**, +not a **peer-graph member**; the connection-local overlay is what +gives the former without the latter. + +### Auth: bearer token on the upgrade request + +Inbound WS auth is `Authorization: Bearer ` on the HTTP upgrade +request, resolved via `IdentityProvider::resolve_from_token()` — the +same path as any HTTP request ([ADR-004](decisions/004-auth-as-shared-core.md); +mechanism owned by alkcall ADR-003). Bearer-only is the auth mechanism; +other schemes would be added as axum middleware on the upgrade route +(two-way door). The resolved identity drives `AccessControl::check` on +every `call.requested` the browser sends and `check_open` on every +channel-open op — per-privilege filtering is free via +`services/list`'s existing `AccessControl` filtering. + +For the consumer side (`from_wss`), the same shape applies outbound: +the adapter presents the token on its dial, sourced from +`Capabilities` (the no-env-vars path, +[ADR-014](decisions/014-secret-material-flow-and-capability-injection.md)). + +## Constraints + +- **The WS path is the channels session, not the gateway shape + ([ADR-067](decisions/067-websocket-carries-channels.md), + [ADR-048](decisions/048-websocket-native-session-not-gateway.md)).** + The 6 gateway endpoints are HTTP-only and do not appear on WS. + Discovery via `services/list`/`services/schema` as call-protocol ops; + streaming as native frames, not SSE. +- **Bearer-only auth on the upgrade request.** `Authorization: Bearer` + → `resolve_from_token`. The resolved identity drives + `AccessControl::check` on calls and `check_open` on channel opens. +- **Browsers are not alk peers.** Bearer token, no `PeerId`, + connection-local Layer 2 overlay for browser-registered ops. +- **One WS binary message = one chunk; channel 0 frames are + length-prefixed JSON inside the chunk payload.** Text messages are + rejected. The alknet "one envelope per message" rule is superseded + ([ADR-048](decisions/048-websocket-native-session-not-gateway.md) + status amendment). +- **The shared `Dispatcher` runs on channel 0 unchanged + (alkcall ADR-015).** The dispatch half is one implementation across + all in-line transports; only connection establishment differs. +- **The default upgrade path is `/alk/channels`; it must not collide + with reserved paths ([ADR-046](decisions/046-assembly-layer-custom-http-routes.md)).** + Overridable via `extra_routes`. +- **The WS↔byte-stream adapter is the single seam between axum's WS + and alkcall's byte-oriented channels machinery** — shared with + `from_wss`, semantics tracked in OQ-01. + +## Design Decisions + +| Decision | ADR | Summary | +|----------|-----|---------| +| WS carries the channels protocol (channel 0 = `alk/call`) | [ADR-067](decisions/067-websocket-carries-channels.md) | WS = in-line channels substrate; 8-byte chunk demux; upgrade path `/alk/channels`; supersedes bare-envelope framing | +| WS carries the native session, not the gateway shape | [ADR-048](decisions/048-websocket-native-session-not-gateway.md) | Channel 0 carries the native call session; gateway endpoints are HTTP-only; discovery via `services/list`/`services/schema` | +| Call protocol stream model | alkcall ADR-015 | Stream-agnostic correlation; the pending map and dispatch loop run over any transport | +| WS as the browser bidirectional path | [ADR-044](decisions/044-defer-webtransport-browsers-use-websocket.md) | Stands; WebTransport removed from alkhttp scope ([ADR-069](decisions/069-webtransport-out-of-scope.md)) | +| Operation registry layering (connection-local overlay) | alkcall ADR-019 | Browser-registered ops land in a per-connection overlay that dies with the WS connection | +| Browsers are not alk peers | [ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) §4 | Bearer token, no `PeerId`; addressability vs bidirectionality rationale | +| Abort cascade on disconnect | alkcall ADR-020 | WS close mid-subscription aborts in-flight ops, cascading to descendants | +| Bearer auth via `resolve_from_token` | [ADR-004](decisions/004-auth-as-shared-core.md) | WS upgrade request credential source (same as HTTP; mechanism: alkcall ADR-003) | +| Browsers require X.509 (TLS) | [ADR-027](decisions/027-tls-identity-redesign-acme-rawkey-decoupling.md) | The WS upgrade runs over the same TLS as h2/http1.1; TLS provisioning is an alknet concern | +| Stealth: HTTP handler on standard ALPNs serves WS upgrade | [ADR-010](decisions/010-alpn-router-and-endpoint.md) | The WS upgrade route is on `HttpAdapter`'s default surface | +| Custom routes collision rule | [ADR-046](decisions/046-assembly-layer-custom-http-routes.md) | The WS upgrade route must not collide with reserved default-surface paths | +| `from_wss` consumer adapter | [ADR-070](decisions/070-from-wss-consumer-adapter.md) | The outbound mirror: same adapter, `ChannelClient` consumer half, `wss` feature | + +## Open Questions + +See [open-questions.md](open-questions.md) for full details. + +- **OQ-01** (open): WS ↔ byte-stream adaptation — buffer bounds, + write-side chunk-completeness assumption (verify against alkcall's + `MuxRunner` atomic chunk writes), flush/close mapping. This is the + implementation risk center for both this path and `from_wss`. +- **OQ-04** (open): browser client library ownership — the JS/TS + client speaking channels-over-WS (chunk framing, channel 0 open ops, + envelope handling) is needed for browser consumers; it lives outside + this crate, but the BAST chunk-header contract + (alkcall `chunk-header.bast.json`) is the shared reference. + +## References + +- [ADR-067](decisions/067-websocket-carries-channels.md) — the + channels-over-WS decision (this document's specification target) +- [ADR-048](decisions/048-websocket-native-session-not-gateway.md) — + the native-session decision (amended by ADR-067 for framing and path) +- [ADR-070](decisions/070-from-wss-consumer-adapter.md) — the consumer + side +- [ADR-044](decisions/044-defer-webtransport-browsers-use-websocket.md) + — WS as the browser path (stands); [ADR-069](decisions/069-webtransport-out-of-scope.md) + — WebTransport out of scope +- [http-server.md](http-server.md) — the `HttpAdapter` hosting the + upgrade route +- The alkcall crate's `docs/architecture/` — call protocol wire format + (ADR-014), stream model (ADR-015), registry layering (ADR-019), abort + cascade (ADR-020), channels wire format (ADR-034/035), channel 0 + pre-negotiation (ADR-036), ChannelsAdapter (ADR-039), ChannelClient + (ADR-043), openable ALPNs (ADR-047); `channels-wire.md` for the + chunk format and wire invariants (REQ-CH-01/02/04/05) +- The alknet mono-repo's `@alkdev/pubsub` WS client/server — prior art + for the browser WS path lineage (the envelope shape the call + protocol refined) \ No newline at end of file diff --git a/src/adapters/mod.rs b/src/adapters/mod.rs index e69de29..8b13789 100644 --- a/src/adapters/mod.rs +++ b/src/adapters/mod.rs @@ -0,0 +1 @@ + diff --git a/src/client/mod.rs b/src/client/mod.rs index e69de29..8b13789 100644 --- a/src/client/mod.rs +++ b/src/client/mod.rs @@ -0,0 +1 @@ + diff --git a/src/gateway/mod.rs b/src/gateway/mod.rs index e69de29..8b13789 100644 --- a/src/gateway/mod.rs +++ b/src/gateway/mod.rs @@ -0,0 +1 @@ + diff --git a/src/lib.rs b/src/lib.rs index 1bc25cf..d994c99 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -6,4 +6,4 @@ pub mod adapters; pub mod client; pub mod gateway; pub mod server; -pub mod websocket; \ No newline at end of file +pub mod websocket; diff --git a/src/server/mod.rs b/src/server/mod.rs index e69de29..8b13789 100644 --- a/src/server/mod.rs +++ b/src/server/mod.rs @@ -0,0 +1 @@ + diff --git a/src/websocket/mod.rs b/src/websocket/mod.rs index e69de29..8b13789 100644 --- a/src/websocket/mod.rs +++ b/src/websocket/mod.rs @@ -0,0 +1 @@ +