docs(research): alknet-http phase-0 findings — HTTP server + client + MCP adapters

Phase 0 exploration for alknet-http (greenfield crate, no existing arch):
HTTP server (axum, ProtocolHandler for h2/http1.1, h3 deferred), HTTP client
(reqwest, the from_openapi/from_mcp forwarding handlers), MCP streamable HTTP
(feature-gated, stdio excluded as security position), to_openapi/to_mcp
projections.

Records: 8 design points (DH-3 HTTP→call operation mapping as the load-bearing
one), the settled adapter location map (from alknet-call gap analysis), the
no-env-vars invariant (Capabilities → from_openapi handler → HTTP header as the
credential injection point), and the prerequisite on alknet-call's
OperationAdapter trait being defined first.
This commit is contained in:
2026-06-25 12:46:25 +00:00
parent 79d8561bb4
commit 6940d9858d

View File

@@ -0,0 +1,381 @@
---
status: draft
last_updated: 2026-06-25
---
# alknet-http — Phase 0 Research Findings
This document captures Phase 0 (Exploration) findings for the `alknet-http`
crate. Unlike alknet-call (which has existing architecture being completed),
alknet-http has **zero architecture docs and zero implementation** — this is a
true greenfield exploration. It will need iteration; the goal here is to surface
the design space, the settled decisions, and the open questions so Phase 1 can
proceed with direction.
## Vision
`alknet-http` is the HTTP protocol handler for the ALPN-as-service architecture.
It serves two roles:
1. **HTTP server** — a `ProtocolHandler` that accepts HTTP/2, HTTP/1.1, and
optionally HTTP/3 (WebTransport) connections on standard ALPNs (`h2`,
`http/1.1`, `h3`), serving REST APIs, dashboards, MCP endpoints, and the
`to_openapi`/`to_mcp` projections of local call-protocol operations.
2. **HTTP client host** — the home of the HTTP-transport-backed adapters
(`from_openapi`, `from_mcp`, `to_openapi`, `to_mcp`) that use reqwest (client)
and axum (server) to bridge between the call protocol and external HTTP/MCP
systems.
The key architectural insight: **HTTP is both a server surface and a client
transport for adapters.** Both directions share the same HTTP dependencies
(axum for serving, reqwest for calling out), which is why they live in one
crate rather than being split.
## Sources Investigated
| Source | Path | Note |
|--------|------|------|
| alknet-core types | `docs/architecture/crates/core/*` | ProtocolHandler, Connection, AuthContext, IdentityProvider |
| alknet-call specs | `docs/architecture/crates/call/*`, ADR-017/022/024 | OperationAdapter trait (where from_openapi/from_mcp implement), HandlerRegistration, Capabilities |
| alknet-call gap analysis | `docs/research/alknet-call-completion/gap-analysis.md` | Adapter location map, the no-env-vars invariant |
| TS @alkdev/operations | `/workspace/@alkdev/operations/src/` | `from_openapi.ts`, `from_mcp.ts`, `from_schema.ts` — prior art for adapter patterns |
| MCP Rust SDK (rmcp) | `/workspace/rust-sdk/` | Streamable HTTP transport (client + server), the `simple_auth_streamhttp.rs` example |
| aisdk | `/workspace/aisdk/` | Rust port of Vercel AI SDK; 75 providers; the no-env-vars problem this crate's credential injection solves |
| overview.md ALPN Registry | `docs/architecture/overview.md` | `h2`/`http/1.1` → HttpAdapter, `h3` → HttpAdapter (WebTransport) |
| endpoint.md stealth section | `docs/architecture/crates/core/endpoint.md` | HTTP handler on standard ALPNs serves decoy — the "stealth mode" mapping |
## What's Settled
These are confirmed by existing ADRs or our prior conversation and should not
be re-litigated in Phase 1.
### 1. alknet-http implements `ProtocolHandler` for standard HTTP ALPNs
From overview.md's ALPN Registry:
| ALPN | Handler | Use case |
|------|---------|----------|
| `h2` | HttpAdapter | HTTP/2 for browsers, curl, standard clients |
| `http/1.1` | HttpAdapter | HTTP/1.1 fallback for legacy clients |
| `h3` | HttpAdapter (WebTransport upgrade) | HTTP/3 / WebTransport for browser streaming |
Unlike custom alknet ALPNs (`alknet/ssh`, `alknet/call`), HTTP uses the
**standard IANA ALPN strings**. This means any HTTP client (browser, curl,
axios) can connect without knowing about alknet — the TLS handshake negotiates
`h2` or `http/1.1` normally, and the HttpAdapter serves HTTP.
### 2. The adapter implementations live here, the trait lives in alknet-call
From the gap analysis adapter location map:
```
alknet-call owns: alknet-http owns:
OperationAdapter trait from_openapi (parse + reqwest forwarding)
from_call (QUIC) to_openapi (generate OpenAPI doc)
from_jsonschema (pure) from_mcp (streamable HTTP client — reqwest)
to_mcp (streamable HTTP server — axum)
```
alknet-http depends on alknet-call for the types (`OperationSpec`, `Handler`,
`HandlerRegistration`, `OperationAdapter`). This is a protocol-foundation
dependency (alknet-call is not a peer handler), within the spirit of ADR-003.
Phase 1 should note this explicitly and possibly amend ADR-003 to clarify
alknet-call's dual role.
### 3. The no-env-vars invariant — credential injection point
The `from_openapi`/`from_mcp` forwarding handlers are the **credential injection
point** for the no-env-vars architecture. The path (from the gap analysis):
```
vault → assembly layer → Capabilities → HandlerRegistration.capabilities
→ OperationContext.capabilities → from_openapi handler reads
context.capabilities.get("openai") → injects into HTTP Authorization
header → reqwest request goes out with vault-derived credential
```
This makes aisdk's `std::env::var("OPENAI_API_KEY")` reads **unreachable**
the assembly layer never calls `Default::default()` on a provider; it
constructs them with vault-derived credentials, or routes HTTP calls through
`from_openapi` operations that carry the credential in `Capabilities`.
**This is a spec-level invariant**: no handler reads outbound credentials from
any source other than `OperationContext.capabilities`. The `from_openapi`/
`from_mcp` implementations in alknet-http are verified against this invariant.
### 4. MCP stdio is explicitly excluded (security position)
From our conversation: MCP stdio transport (`transport-child-process` in rmcp)
= spawn an arbitrary executable, pipe JSON-RPC over stdin/stdout. Downloading
untrusted MCP servers and running them via stdio is indistinguishable from
`curl | sh` with extra steps. The "download untrusted and probably poorly
written mcp servers" model is an RCE vector.
**alknet-http supports only streamable HTTP for MCP.** Stdio is not built. If
someone wants stdio MCP, they run it themselves outside alknet. This is an
explicit security position, recorded as an ADR-level constraint in the
alknet-http spec. The streamable HTTP transport is the supported path; it's
network-isolated, auth-gatable (Bearer token middleware, per the rmcp
`simple_auth_streamhttp.rs` example), and runs under alknet's auth/identity/
capabilities machinery.
### 5. Stealth mode = HTTP handler on standard ALPNs
From endpoint.md: the reference implementation's "stealth mode" (SSH-over-TLS
on port 443 with a fake nginx 404 for non-SSH traffic) maps to the HTTP handler
serving a decoy website or fake 404 on standard HTTP ALPNs. Clients that don't
offer alknet ALPNs get the HTTP handler — just like port scanners in stealth
mode. No byte-peeking; ALPN does the routing.
## Design Space (Decision Points)
These are genuine architectural choices for a greenfield crate. Each is tagged
with door type per ADR-009. Phase 1 resolves these (some via ADRs, some as
two-way-door defaults).
### DH-1: HTTP framework — axum
*(Recommended: two-way door — axum is the obvious choice but not locked)*
The overview.md and the rmcp examples all use axum. axum is built on hyper,
integrates with tower (middleware), and is the de facto Rust web framework.
Alternatives (actix-web, warp, raw hyper) exist but axum is the path of least
resistance and aligns with the rmcp streamable HTTP server (which uses axum's
`Router` + `StreamableHttpService`).
**Recommendation**: **axum**. The rmcp `StreamableHttpService` is a tower
service that nests into an axum `Router` directly (see
`simple_auth_streamhttp.rs:134-159`), so using axum makes the MCP server
integration trivial. Two-way door — switching frameworks later is painful but
not architecturally locked.
### DH-2: HTTP/3 + WebTransport — in v1 or deferred?
*(Recommended: two-way door — defer h3/WebTransport past v1)*
HTTP/3 (QUIC-based) and WebTransport are the browser-streaming path. Browsers
don't support RFC 7250 raw keys (ADR-027), so WebTransport requires X.509
certs — meaning it's a domain-hosted-service concern, not a P2P concern.
quinn already speaks QUIC; HTTP/3 is a framing layer on top.
The question is whether v1 needs WebTransport or whether HTTP/2 + HTTP/1.1
suffices. For the runner pattern and the call-protocol HTTP projection,
HTTP/2 is sufficient. WebTransport is the browser path for the agent service
(alknet-agent), which is downstream of alknet-http anyway.
**Recommendation**: **defer h3/WebTransport past v1**. Start with `h2` +
`http/1.1`. The `h3` ALPN is reserved in the registry but the implementation
lands as a fast-follow when the agent service needs browser streaming. This
keeps v1 focused on the adapter + REST surface. Two-way door.
### DH-3: How does HTTP map to call-protocol operations?
*(One-way door — needs an ADR)*
The core question: when an HTTP request arrives at `alknet/http://api.example.com/container/exec`,
how does it become a call-protocol operation? Options:
- **(a) Direct path mapping**: `POST /{service}/{op}``call.requested` for
`/{service}/{op}`. Matches the call protocol's `/{service}/{op}` path format
(OQ-13). The HTTP handler is a thin bridge: parse the HTTP request body as
the operation input, send `call.requested`, return the response as JSON.
- **(b) OpenAPI-defined routes**: the HTTP surface is defined by the
`to_openapi` projection — routes, methods, schemas are generated from the
registry's `External` operations. The HTTP handler dispatches based on the
generated OpenAPI spec's path mapping.
- **(c) Explicit route registration**: the assembly layer registers HTTP routes
explicitly, mapping URL paths to operations. Most flexible, most boilerplate.
**Recommendation**: **(a) direct path mapping as the default, with (b) as the
discovery/projection layer**. The HTTP handler receives `POST /container/exec`,
constructs `{operationId: "container/exec", input: <parsed body>}`, and
dispatches it through the call protocol. `to_openapi` generates the spec that
*describes* this surface for external consumers; it doesn't define separate
routes. This keeps the HTTP surface a thin projection of the call protocol,
not a parallel routing layer. Needs an ADR.
### DH-4: Auth — how does HTTP auth map to AuthContext?
*(One-way door — needs an ADR, but largely settled by existing ADRs)*
The `HttpAdapter` extracts credentials and resolves identity through
`IdentityProvider` (ADR-004). The credential source for HTTP is the
`Authorization: Bearer <token>` header. The handler calls
`resolve_from_token(&AuthToken { raw: token_bytes })`. This is already in
auth.md's table:
| Handler | Credential source | Resolution method |
|---------|------------------|-----------------|
| HttpAdapter | `Authorization: Bearer` header | `resolve_from_token()` |
**Recommendation**: this is settled by ADR-004 + auth.md. The `HttpAdapter`
constructor-injects `Arc<dyn IdentityProvider>` (same pattern as `SshAdapter`).
No new ADR needed — Phase 1 documents the existing model. The one sub-question
is whether HTTP supports multiple auth mechanisms (Bearer + basic + API key in
query param) or Bearer-only. **Recommendation: Bearer-only for v1** (matches
the call protocol's AuthToken model); other mechanisms are two-way-door
additions.
### DH-5: MCP streamable HTTP — server and/or client?
*(Recommended: both, feature-gated — confirmed in conversation)*
From our conversation + rmcp survey:
- **`from_mcp`** (import remote MCP tools): uses rmcp's
`StreamableHttpClientTransport` (reqwest-based,
`transport-streamable-http-client-reqwest` feature). Discovers MCP tools,
registers them as `FromMCP`-provenance operations with forwarding handlers.
- **`to_mcp`** (expose local ops as MCP tools): uses rmcp's
`StreamableHttpService` (axum-based,
`transport-streamable-http-server` feature). Serves local `External`
operations as MCP tools over streamable HTTP.
Both are feature-gated (the rmcp dependency is optional). The MCP server
auth uses Bearer token middleware (the `simple_auth_streamhttp.rs` example
shows the pattern).
**Recommendation**: **both, behind an `mcp` feature gate**. The rmcp
dependency (`rmcp = { version = "1.8", features = [...] }`) is optional;
enabling the `mcp` feature pulls in rmcp with the streamable HTTP transport
features. Without the feature, alknet-http is a pure HTTP server + OpenAPI
adapter with no MCP support. Two-way door on the feature gate; one-way door
on stdio exclusion.
### DH-6: Dashboard / health / metrics surface
*(Two-way door — start minimal, extend later)*
An HTTP server typically needs operational endpoints: health check, metrics,
maybe a dashboard. The question is whether these are call-protocol operations
(`/health/check`, `/metrics/list`) or raw HTTP routes outside the call
protocol.
**Recommendation**: **health check as a raw HTTP route (`GET /healthz`)**
outside the call protocol (no auth, no operation registration — it's an
infrastructure endpoint for load balancers). Metrics and dashboard are
call-protocol operations (`/metrics/list`, `/dashboard/view`) if built at all.
Start with just `/healthz` in v1. Two-way door.
### DH-7: HTTP client — reqwest config and connection pooling
*(Two-way door — implementation detail)*
The `from_openapi`/`from_mcp` forwarding handlers use reqwest to make outbound
HTTP calls. The aisdk `core/client.rs` shows a pattern worth referencing: a
shared `reqwest::Client` with connection pooling (`OnceLock<reqwest::Client>`),
retry logic (exponential backoff, Retry-After header), and separate streaming
vs non-streaming clients.
**Recommendation**: alknet-http maintains its own shared `reqwest::Client`
(constructed once, reused across all forwarding handlers). The retry/pooling
config comes from `StaticConfig` or `DynamicConfig` (hot-reloadable). Don't
inherit aisdk's client — alknet-http owns its HTTP client. The credential
injection happens per-request (from `OperationContext.capabilities`), not at
client construction. Two-way door on the exact pooling/retry config.
### DH-8: TLS for outbound HTTP calls
*(Two-way door — implementation detail, connects to ADR-027)*
The `from_openapi`/`from_mcp` handlers make outbound HTTPS calls. The TLS
config for these calls: do they use the system trust store (default reqwest
behavior), or a custom CA bundle, or vault-derived client certs?
**Recommendation**: **system trust store by default** (standard HTTPS to
external APIs like OpenAI, Anthropic, etc.). Custom CA bundle + client certs
as an optional config for self-hosted API gateways. This is a two-way-door
implementation detail; the credential (API key/token) comes from
`Capabilities`, the TLS trust comes from the system. No ADR needed.
## Tentative Recommended Approach (Convergence)
1. **Crate**: `alknet-http`, depends on `alknet-core` (ProtocolHandler,
AuthContext, IdentityProvider), `alknet-call` (OperationAdapter trait,
OperationSpec, HandlerRegistration, Capabilities), `axum` (HTTP server),
`reqwest` (HTTP client), and optionally `rmcp` (MCP, feature-gated).
2. **HTTP server** (`h2` + `http/1.1`): axum `Router` wrapped as a
`ProtocolHandler`. Receives the QUIC `Connection`, accepts a bistream,
hands the duplex stream to hyper/axum's connection handler. Direct path
mapping (`POST /{service}/{op}``call.requested`) for the default
surface. Bearer auth via `IdentityProvider::resolve_from_token()`.
3. **HTTP client** (`from_openapi`/`from_mcp` forwarding): shared
`reqwest::Client` with pooling + retry. Credentials from
`OperationContext.capabilities` per the no-env-vars invariant. The
`from_openapi` handler reads `context.capabilities.get("openai")` and
injects into the `Authorization` header.
4. **MCP** (feature-gated `mcp`): `from_mcp` via rmcp
`StreamableHttpClientTransport` (reqwest); `to_mcp` via rmcp
`StreamableHttpService` (axum). Stdio excluded (security position).
5. **`to_openapi`**: generates an OpenAPI spec from the local registry's
`External` operations. Pure serialization — no HTTP client, no HTTP server.
Served at `GET /openapi.json` (or similar) by the HTTP server.
6. **`h3`/WebTransport**: deferred past v1. ALPN reserved, implementation
lands as a fast-follow for the agent service's browser-streaming path.
7. **Health**: `GET /healthz` as a raw HTTP route (no auth, no call protocol).
8. **Stealth**: the HTTP handler on `h2`/`http/1.1` serves a decoy/fake 404
for unknown paths, matching the endpoint.md stealth-mode mapping. Real
services use `alknet/ssh`, `alknet/call`, etc.
## Open Questions to Carry into Phase 1
- **OQ-HTTP-01 (HTTP → call operation mapping)**: direct path mapping
(`POST /{service}/{op}``call.requested`) as the default — confirmed
approach, needs an ADR (DH-3).
- **OQ-HTTP-02 (h3/WebTransport timeline)**: deferred past v1 (DH-2). Two-way
door; lands when the agent service needs browser streaming.
- **OQ-HTTP-03 (MCP feature gate shape)**: the exact rmcp features to enable
and the `mcp` feature gate definition (DH-5). Two-way door.
- **OQ-HTTP-04 (dashboard/metrics surface)**: minimal `/healthz` in v1,
metrics/dashboard as call-protocol operations if built (DH-6). Two-way door.
- **OQ-HTTP-05 (ADR-003 amendment)**: clarify that alknet-http depending on
alknet-call is permitted (alknet-call is protocol-foundation, not a peer
handler). One-line amendment.
- **OQ-HTTP-06 (to_openapi published-spec versioning)**: ADR-017 Consequences
notes that published `to_*` specs are compatibility contracts. The versioning
strategy for generated OpenAPI specs (tied to the registry's External
operation set version) needs specifying. One-way door after first
publication.
## Next Steps (Phase 0 → Phase 1)
1. **You decide** on DH-3 (HTTP → call operation mapping) — this is the
load-bearing architectural choice. The others (DH-1, DH-2, DH-4, DH-5,
DH-6, DH-7, DH-8) are two-way-door defaults I recommend accepting as-is.
2. **alknet-call completion prerequisite**: the `OperationAdapter` trait must
be defined in alknet-call before alknet-http's `from_openapi`/`from_mcp`
can be implemented. Phase 1 for alknet-http can proceed in parallel with
alknet-call completion (the spec can be written against the trait shape from
ADR-017), but implementation of the HTTP-backed adapters blocks on the
trait landing.
3. **Phase 1 (Architect)**: produce `docs/architecture/crates/http/README.md`
+ component specs (e.g., `http-server.md`, `http-client.md`,
`http-adapters.md`, `http-mcp.md`), ADRs for DH-3 (HTTP→call mapping) and
the MCP stdio exclusion (security position), and the OQs above. Update
`docs/architecture/README.md` index and ADR table.
## References
- `docs/sdd_process.md` — Phase 0 process definition
- `docs/architecture/overview.md` — ALPN Registry (h2/http1.1/h3 → HttpAdapter)
- `docs/architecture/crates/core/core-types.md` — ProtocolHandler, Connection
- `docs/architecture/crates/core/auth.md` — HttpAdapter auth (Bearer → resolve_from_token)
- `docs/architecture/crates/core/endpoint.md` — stealth mode as ALPN dispatch
- `docs/architecture/decisions/017-call-protocol-client-and-adapter-contract.md`
OperationAdapter trait, to_openapi/to_mcp
- `docs/architecture/decisions/022-handler-registration-provenance-and-composition-authority.md`
Capabilities injection, FromOpenAPI/FromMCP provenance
- `docs/architecture/decisions/027-tls-identity-redesign-acme-rawkey-decoupling.md`
browser limitation (no RFC 7250), WebTransport needs X.509
- `docs/research/alknet-call-completion/gap-analysis.md` — adapter location map,
no-env-vars invariant, exchange-of-operations pattern
- `/workspace/@alkdev/operations/src/` — TS prior art: `from_openapi.ts`,
`from_mcp.ts`, `from_schema.ts`, `scanner.ts`
- `/workspace/rust-sdk/` — MCP Rust SDK (rmcp v1.8.0); streamable HTTP transport
- `/workspace/rust-sdk/examples/servers/src/simple_auth_streamhttp.rs`
streamable HTTP MCP server with Bearer auth (the pattern for `to_mcp`)
- `/workspace/rust-sdk/examples/clients/src/streamable_http.rs`
streamable HTTP MCP client (the pattern for `from_mcp`)
- `/workspace/aisdk/` — Rust port of Vercel AI SDK; 75 providers with env-var
reads that the no-env-vars invariant makes unreachable
- `/workspace/aisdk/src/core/client.rs` — HTTP client reference (pooling,
retry, streaming vs non-streaming)