glm-5.3-flash
4a825d33e7
Full-surface integration suite (tests/full_surface.rs, mcp feature): - one HttpAdapter over real TCP (ProtocolHandler::handle path) serving gateway endpoints, /openapi.json, /mcp, and the WS channels session - gateway: search/schema/call/subscribe/batch/publish presence, envelope shapes, error fidelity end-to-end - from_openapi import -> Internal-by-default invisible from the wire -> External facade composes it via env.invoke -> upstream HTTP API called end-to-end (ADR-015 composition model exercised) - to_openapi 6-path doc validated against openapiv3 over the wire - to_mcp: MCP client connects to /mcp on the served adapter, lists the 4 gateway tools, search returns ACL-filtered ops (Sub excluded) Production fix: the WS upgrade route was reserved but never wired into HttpAdapter's router (the ws-upgrade-session tests built their own router). Now wired with ws_bearer_auth (401 without a resolvable token) around ws_upgrade_handler. Docs sync: all 28 'Port notes' sections/blockquotes stripped from ported ADRs/specs; OQ-01/OQ-02 statuses corrected to resolved in overview.md, websocket.md, and the README table (open-questions.md was already current). Publish prep: cargo publish --dry-run --allow-dirty succeeds; cargo doc --no-deps warning-free (ADR link targets fixed); feature combinations (default / test-support / mcp / wss / all) compile warning-free under clippy -D warnings. Verified: cargo test (182 lib default), --all-features (227 lib + 29 integration), clippy -D warnings x3 feature sets, fmt, doc, publish --dry-run.
305 lines
18 KiB
Markdown
305 lines
18 KiB
Markdown
---
|
|
status: draft
|
|
last_updated: 2026-08-27
|
|
---
|
|
|
|
# alkhttp — Overview
|
|
|
|
The HTTP interface crate: serves inbound HTTP on standard ALPNs (with
|
|
WebSocket upgrade carrying the channels protocol for browser
|
|
bidirectional access) and hosts the HTTP-backed call-protocol adapters.
|
|
This document covers the crate's two roles, its dependency edges, and
|
|
the adapter location map. Component details are in the sibling
|
|
documents.
|
|
|
|
## What
|
|
|
|
alkhttp is the HTTP protocol handler crate for the alk stack. It serves
|
|
two roles in one crate ([ADR-039](decisions/039-http-server-and-client-host-colocated.md)):
|
|
|
|
1. **HTTP server** — a `ProtocolHandler` (`HttpAdapter`) that accepts
|
|
HTTP/2 and HTTP/1.1 connections on the standard IANA ALPNs (`h2`,
|
|
`http/1.1`), plus WebSocket upgrade (for browser bidirectional
|
|
access to the call protocol). It serves REST APIs, the
|
|
`to_openapi`/`to_mcp` projections of local call-protocol operations,
|
|
the `/healthz` operational endpoint, and the decoy surface for
|
|
stealth mode. The WS path carries the **channels protocol**
|
|
([ADR-067](decisions/067-websocket-carries-channels.md)): the 8-byte
|
|
chunk header multiplexes N channels over the WS connection; channel 0
|
|
is pre-negotiated as `alk/call` and runs the shared `Dispatcher`.
|
|
2. **HTTP client host** — the home of the HTTP-transport-backed call
|
|
adapters: `from_openapi` (import external HTTP APIs described by an
|
|
OpenAPI document, using `reqwest` for outbound calls), `from_mcp`
|
|
(import remote MCP tools over streamable HTTP, using `reqwest`),
|
|
`from_jsonschema` (import a single non-OpenAPI HTTP endpoint), and
|
|
`from_wss` (import a remote alk node's operations over a WSS
|
|
channels connection — same-protocol importer). The reverse
|
|
projections `to_openapi` (generate an OpenAPI doc from the local
|
|
registry's `External` operations) and `to_mcp` (expose local ops as
|
|
MCP tools over streamable HTTP, using `axum`) also live here.
|
|
|
|
Both directions share the same HTTP dependencies (`axum` for serving,
|
|
`reqwest` for calling out), which is why they live in one crate rather
|
|
than being split into a server crate and a client crate.
|
|
|
|
## Why
|
|
|
|
The crate's purpose is to be the HTTP interface library for downstream
|
|
crates that need to expose an HTTP interface. A downstream consumer (a
|
|
hub deployment, a browser-facing service, the alknet assembly layer)
|
|
wires `HttpAdapter` into the handler registry for the standard HTTP
|
|
ALPNs and gets a full HTTP surface: the gateway projection of the call
|
|
protocol, OpenAPI discovery, MCP tool exposure, and WebSocket for
|
|
browser bidirectional access to both the call protocol and data
|
|
channels.
|
|
|
|
### The producer/consumer model
|
|
|
|
The call protocol is bidirectional — both sides can initiate calls —
|
|
and the channels protocol is symmetric — either side can open
|
|
channels. **Producer** and **consumer** therefore describe *what a side
|
|
does on a given operation or channel*, not what it is as a connection
|
|
endpoint:
|
|
|
|
- A **producer** (call) registers operations on an `OperationRegistry`
|
|
and runs a `Dispatcher`; a **consumer** (call) uses a `CallConnection`
|
|
to invoke them. On one connection both sides can be producers of some
|
|
ops and consumers of others.
|
|
- A **producer** (channels) accepts data channels (registers openable
|
|
ALPNs); a **consumer** (channels) opens them via the per-ALPN open
|
|
ops on channel 0. Both sides can open and accept on the same
|
|
connection.
|
|
|
|
The HTTP surface maps onto this model as follows:
|
|
|
|
- The `HttpAdapter` (gateway + WS upgrade) is the **producer-facing
|
|
surface**: it runs the shared `Dispatcher` on channel 0 for WS
|
|
sessions, and the gateway dispatches into the local registry for HTTP
|
|
requests. A browser or HTTP client is a **consumer** of those ops.
|
|
- The WS session is fully bidirectional: the browser side can also be a
|
|
producer (registering ops via the connection-local Layer 2 overlay,
|
|
opening channels), making the hub a consumer of browser ops on the
|
|
same connection.
|
|
- The `from_*` adapters are **consumer-side**: they import external
|
|
surfaces (OpenAPI, MCP, WSS) into the local registry as ops the local
|
|
process can then consume or compose.
|
|
- The `to_*` adapters are **producer-side projections**: they expose
|
|
the local registry's ops through foreign protocols (OpenAPI, MCP).
|
|
|
|
Connection direction (who opened the HTTP/TCP connection) is
|
|
independent of these roles, exactly as in the call protocol.
|
|
|
|
A note on the "from/to" direction model: the `from_openapi`/`to_openapi`
|
|
and `from_mcp`/`to_mcp` adapters are *inherently directional* because
|
|
OpenAPI and MCP are client/server protocols — one side serves, the
|
|
other calls. That directionality is a property of those protocols, not
|
|
of the call protocol itself. The call protocol is bidirectional. The
|
|
HTTP/1.1 + HTTP/2 surface inherits HTTP's request/response constraint
|
|
and projects the call protocol one-directionally (consumer→producer
|
|
calls only). **WebSocket restores the call protocol's native
|
|
bidirectionality for browsers** — and now carries the channels protocol
|
|
besides it.
|
|
|
|
## Dependencies
|
|
|
|
```
|
|
alkhttp
|
|
├── alkcall (call + channels protocol, vendored core types,
|
|
│ registry, adapter contract — the sole alk dep)
|
|
├── axum (HTTP server — Router, extractors, middleware, WebSocket upgrade)
|
|
├── hyper / hyper-util(hTTP/1.1 + HTTP/2 framing; axum is built on hyper)
|
|
├── reqwest (HTTP client — from_openapi/from_mcp forwarding)
|
|
│ + reqwest-middleware, reqwest-retry
|
|
├── tokio-tungstenite (WSS client — from_wss; behind the `wss` feature)
|
|
├── yaml_serde (YAML parse for from_openapi YAML input — ADR-051)
|
|
├── openapiv3 (OpenAPI doc model for from_openapi/to_openapi)
|
|
└── rmcp (MCP streamable HTTP — feature-gated behind `mcp`)
|
|
```
|
|
|
|
> **Note:** the `h3`/WebTransport dependency is **not** in the
|
|
> dependency tree and is not planned — WebTransport is out of scope in
|
|
> alkhttp ([ADR-069](decisions/069-webtransport-out-of-scope.md)); when
|
|
> a browser needs WebTransport it will be provided by the alknet layer,
|
|
> which owns transports. The browser bidirectional path is WebSocket
|
|
> (native axum support, no new dependency).
|
|
|
|
### The alkcall dependency ([ADR-003](decisions/003-crate-decomposition.md) Amendment 1)
|
|
|
|
alkhttp depends on alkcall. The old rule "no handler crate depends on
|
|
another handler crate" survives as "no HTTP adapter depends on another
|
|
transport adapter": alkcall is the protocol-foundation crate (it owns
|
|
the vendored core types — `Connection`, `ProtocolHandler`, `BiStream`,
|
|
`AuthContext`, `IdentityProvider`, `Capabilities`, `OwnershipProvider`,
|
|
`HandlerError`, `StreamError` — plus the registry, dispatch, and wire
|
|
format). alkhttp depending on alkcall is "HTTP uses the call protocol
|
|
types," not "HTTP depends on another transport."
|
|
|
|
alkcall stays transport-agnostic — no `reqwest`, no `axum`, no HTTP
|
|
dependencies. The `from_openapi`/`from_mcp`/`from_jsonschema`/`from_wss`
|
|
forwarding handlers are opaque `Arc<dyn Handler>`-shaped registrations
|
|
from the registry's perspective: constructed by alkhttp at registration
|
|
time, stored in `HandlerRegistration`, dispatched by the `Dispatcher`
|
|
which doesn't know `reqwest` or `tokio-tungstenite` is involved.
|
|
|
|
## ALPNs
|
|
|
|
| ALPN | Handler | Transport | Browser? |
|
|
|------|---------|-----------|----------|
|
|
| `http/1.1` | `HttpAdapter` | HTTP/1.1 over a `BiStream` (+ WS upgrade) | Yes (WS upgrade for bidirectional) |
|
|
| `h2` | `HttpAdapter` | HTTP/2 over a `BiStream` (+ WS upgrade) | Yes (WS upgrade for bidirectional) |
|
|
| `h3` | — | — | Out of scope ([ADR-069](decisions/069-webtransport-out-of-scope.md)); an alknet concern |
|
|
|
|
These are standard IANA ALPN strings, not `alk/`-prefixed. Any HTTP
|
|
client connects without knowing about the alk stack — the TLS handshake
|
|
negotiates `h2` or `http/1.1` normally, and the `HttpAdapter` serves
|
|
HTTP. This is the stealth mapping
|
|
([ADR-010](decisions/010-alpn-router-and-endpoint.md)).
|
|
|
|
The `HttpAdapter` registers for `http/1.1` and `h2`. The endpoint's
|
|
handler registry maps each ALPN to the same adapter instance; the
|
|
handler branches on `connection.remote_alpn()` to pick the framing.
|
|
WebSocket upgrade rides on either HTTP version (RFC 6455 over HTTP/1.1;
|
|
extended CONNECT over HTTP/2).
|
|
|
|
## Adapter Location Map
|
|
|
|
The decomposition principle: the adapter trait lives where the types
|
|
live (alkcall); the adapter implementations live where their transport
|
|
dependencies live (alkhttp).
|
|
|
|
```
|
|
alkcall (transport-agnostic — no HTTP client, no HTTP server)
|
|
├── OperationAdapter trait (the contract — async)
|
|
├── from_call (same-protocol importer over a call connection)
|
|
├── CallConnection / CallClient (outbound call surface)
|
|
└── ChannelsAdapter / ChannelClient (channels protocol, both sides)
|
|
|
|
alkhttp (owns HTTP server + HTTP client + WSS client)
|
|
├── HttpAdapter (axum server — inbound HTTP on h2/http1.1 + WS upgrade route)
|
|
├── [WS upgrade → channels session] (not an adapter — hands the WS byte stream to the
|
|
│ channels machinery; see websocket.md, ADR-067)
|
|
├── from_openapi (parse OpenAPI doc + reqwest forwarding handler)
|
|
├── from_jsonschema (single-endpoint reqwest forwarding handler — ADR-066)
|
|
├── to_openapi (generate OpenAPI doc from local registry — the gateway projection)
|
|
├── from_mcp (feature-gated) (import remote MCP tools over streamable HTTP — reqwest)
|
|
├── to_mcp (feature-gated) (expose local ops as MCP tools over streamable HTTP — axum)
|
|
└── from_wss (feature-gated) (import a remote node's ops over WSS — ADR-070)
|
|
```
|
|
|
|
alkcall never sees the HTTP client. The forwarding handlers are opaque
|
|
from the registry's perspective. alkcall stays lean; alkhttp owns all
|
|
HTTP directions and the WSS consumer.
|
|
|
|
## Feature Gates
|
|
|
|
```toml
|
|
[features]
|
|
default = ["h2", "http1"] # the HTTP surface (incl. WebSocket upgrade for browsers)
|
|
mcp = ["dep:rmcp"] # from_mcp / to_mcp (streamable HTTP only — ADR-037)
|
|
wss = ["dep:tokio-tungstenite"] # from_wss consumer adapter (ADR-070)
|
|
```
|
|
|
|
- `h2` + `http1` (default): the `axum` + `hyper` HTTP/1.1 + HTTP/2
|
|
server, including WebSocket upgrade for browser bidirectional access.
|
|
This is the surface all clients — including browsers, via WS upgrade —
|
|
use.
|
|
- `mcp`: the `rmcp` dependency with streamable HTTP transport features
|
|
only. Adds `from_mcp`/`to_mcp`. See [http-mcp.md](http-mcp.md) and
|
|
[ADR-037](decisions/037-mcp-stdio-transport-exclusion.md).
|
|
- `wss`: the `tokio-tungstenite` dependency. Adds `from_wss`. See
|
|
[ADR-070](decisions/070-from-wss-consumer-adapter.md).
|
|
|
|
**`yaml_serde` is not feature-gated** — YAML OpenAPI is a first-class
|
|
input format (some providers publish YAML-only schemas), not an edge
|
|
case ([ADR-051](decisions/051-yaml-input-for-from-openapi.md) §3).
|
|
|
|
## The No-Env-Vars Invariant
|
|
|
|
The `from_openapi`/`from_mcp`/`from_jsonschema`/`from_wss` forwarding
|
|
handlers are the **credential injection point** for the no-env-vars
|
|
architecture. The path:
|
|
|
|
```
|
|
vault (alkvault) → assembly layer → Capabilities
|
|
→ HandlerRegistration.capabilities → OperationContext.capabilities
|
|
→ handler reads context.capabilities.get("<service>")
|
|
→ injects into outbound HTTP Authorization header (or WSS auth)
|
|
```
|
|
|
|
This makes any downstream `std::env::var("OPENAI_API_KEY")` read
|
|
unreachable — the assembly layer never calls `Default::default()` on a
|
|
provider; it constructs them with vault-derived credentials, or routes
|
|
outbound calls through adapter operations that carry the credential in
|
|
`Capabilities`.
|
|
|
|
**This is a spec-level invariant**: no handler reads outbound
|
|
credentials from any source other than `OperationContext.capabilities`.
|
|
All adapter implementations in alkhttp are verified against this
|
|
invariant. See
|
|
[ADR-014](decisions/014-secret-material-flow-and-capability-injection.md).
|
|
|
|
## Architecture (component pointers)
|
|
|
|
- **[http-server.md](http-server.md)** — the `HttpAdapter` for `h2`/
|
|
`http/1.1` (+ the WS upgrade route): how axum is run over a
|
|
`BiStream`, Bearer auth resolution, the `/healthz` raw route, stealth
|
|
decoy, the 6-endpoint gateway dispatch, and the WS upgrade route
|
|
(which hands off to the channels session).
|
|
- **[websocket.md](websocket.md)** — the WebSocket browser bidirectional
|
|
path: the channels protocol over WS, framing via the WS↔byte-stream
|
|
adapter, channel 0 and the shared `Dispatcher`, bidirectionality,
|
|
connection-local Layer 2 overlay, the browsers-are-not-peers
|
|
rationale, streaming (native, no SSE).
|
|
- **[http-adapters.md](http-adapters.md)** — `from_openapi`,
|
|
`from_jsonschema`, `to_openapi`, and `from_wss`. Error fidelity per
|
|
[ADR-023](decisions/023-operation-error-schemas.md).
|
|
- **[http-mcp.md](http-mcp.md)** — `from_mcp`/`to_mcp` (feature-gated),
|
|
streamable HTTP only (ADR-037), the rmcp integration.
|
|
|
|
## Design Decisions
|
|
|
|
| Decision | ADR | Summary |
|
|
|----------|-----|---------|
|
|
| HTTP-to-call operation mapping | [ADR-036](decisions/036-http-to-call-operation-mapping.md) | ~~Direct path mapping~~ — **routing superseded by ADR-047**; non-routing clauses survive (SSE, auth, `/healthz`, stealth, error mapping) |
|
|
| MCP stdio transport exclusion | [ADR-037](decisions/037-mcp-stdio-transport-exclusion.md) | Streamable HTTP only; stdio is not built (RCE vector) |
|
|
| WebSocket carries the channels protocol | [ADR-067](decisions/067-websocket-carries-channels.md) | WS = in-line channels substrate; channel 0 = `alk/call`; upgrade path `/alk/channels`; new in alkhttp |
|
|
| Gateway `/publish` endpoint | [ADR-068](decisions/068-gateway-publish-endpoint.md) | 6th gateway endpoint for `Pub` ops; NDJSON request body → `call.published` chunks; new in alkhttp |
|
|
| WebTransport out of scope | [ADR-069](decisions/069-webtransport-out-of-scope.md) | h3/WebTransport removed from alkhttp scope; an alknet concern; supersedes ADR-044's deferral framing |
|
|
| `from_wss` consumer adapter | [ADR-070](decisions/070-from-wss-consumer-adapter.md) | Same-protocol importer over WSS; `wss` feature; new in alkhttp |
|
|
| WS carries the native session, not the gateway shape | [ADR-048](decisions/048-websocket-native-session-not-gateway.md) | Amended by ADR-067: channel 0 carries the native call session; gateway endpoints are HTTP-only |
|
|
| HTTP server + client host colocated | [ADR-039](decisions/039-http-server-and-client-host-colocated.md) | One crate for server + adapters (shared HTTP deps, shared mapping) |
|
|
| `to_mcp` tool-gateway pattern | [ADR-041](decisions/041-mcp-tool-gateway-pattern.md) | 4 fixed gateway tools (search/schema/call/batch); Sub and Pub excluded |
|
|
| `to_openapi` gateway pattern | [ADR-042](decisions/042-openapi-gateway-pattern.md), [ADR-047](decisions/047-remove-direct-call-http-surface.md) | Fixed gateway endpoints are the sole HTTP invoke path (no per-operation `POST /{service}/{op}`); per-caller AccessControl-filtered `/search` is the discovery; extended with `/publish` (ADR-068) |
|
|
| Assembly-layer custom HTTP routes | [ADR-046](decisions/046-assembly-layer-custom-http-routes.md) | `extra_routes: Option<Router>` at construction; default surface takes precedence on collision |
|
|
| alkhttp is protocol-foundation-dependent | [ADR-003](decisions/003-crate-decomposition.md) Am. 1 | alkhttp depends on alkcall alone (types, not a peer transport) |
|
|
| Bearer auth via `resolve_from_token` | [ADR-004](decisions/004-auth-as-shared-core.md) | HTTP handler credential source + resolution (settled; alkcall ADR-003 owns the mechanism) |
|
|
| Stealth mode = HTTP handler on standard ALPNs | [ADR-010](decisions/010-alpn-router-and-endpoint.md) | Decoy for unknown paths (settled) |
|
|
| Adapter-registered ops are `Internal` | [ADR-015](decisions/015-privilege-model-and-authority-context.md) | `from_openapi`/`from_mcp`/`from_jsonschema`/`from_wss` produce `Internal` leaves (record: alkcall ADR-017) |
|
|
| `OperationAdapter` trait is async | [ADR-017](decisions/017-call-protocol-client-and-adapter-contract.md) | HTTP adapters implement the async trait (record: alkcall ADR-022) |
|
|
| `to_*` adapters are projections | [ADR-017](decisions/017-call-protocol-client-and-adapter-contract.md) | `to_openapi`/`to_mcp` consume the registry, don't produce entries (record: alkcall ADR-022) |
|
|
| Error schema fidelity | [ADR-023](decisions/023-operation-error-schemas.md) | `from_openapi` maps HTTP status → `HTTP_<status>` codes; `to_openapi` projects back (record: alkcall ADR-016) |
|
|
| Browsers require X.509 | [ADR-027](decisions/027-tls-identity-redesign-acme-rawkey-decoupling.md) | Browser-facing TLS uses X.509 (TLS provisioning itself is an alknet concern) |
|
|
| Browsers are not alk peers | [ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) §4 | Browser over WS = bearer token, no `PeerId` |
|
|
|
|
## Open Questions
|
|
|
|
See [open-questions.md](open-questions.md) for full details.
|
|
|
|
- **OQ-01** (resolved): WS message ↔ byte-stream adaptation — the
|
|
production adapter (`src/websocket/byte_adapter.rs`) is validated in
|
|
both directions (server upgrade path + `from_wss` client).
|
|
- **OQ-02** (resolved): `/publish` body framing — first line
|
|
`{operation, chunk}`; terminal errors as plain HTTP status + JSON
|
|
body (ADR-068).
|
|
- **OQ-03** (open): `from_wss` reconnection semantics — v1 = drop →
|
|
retryable failures (ADR-070); policy deferred to the assembly layer.
|
|
|
|
## References
|
|
|
|
- The alkcall crate — `docs/architecture/` for the call and channels
|
|
protocol decisions; `README.md` §"Roles and Composition" for the
|
|
producer/consumer role definitions.
|
|
- The alkvault crate — secrets, feeding `Capabilities` at the assembly
|
|
layer.
|
|
- The alknet mono-repo — endpoint/transport concerns and the source
|
|
specs this crate was ported from. |