diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 38d6cfd..d8a2089 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-07-08 +last_updated: 2026-07-09 --- # Alknet Architecture @@ -26,6 +26,25 @@ The alknet-call crate is **implemented and reviewed** — both the server-side c **alknet-docker specs drafted.** The alknet-docker crate (docker operations on the shared `alknet/call` ALPN + `DockerTtyBackend` behind a `tty` feature) now has architecture specs: [crates/docker/](crates/docker/) (overview, docker-operations, docker-tty-backend) and six ADRs — [ADR-058](decisions/058-alknet-docker-on-alknet-call.md) (docker ops register on `alknet/call`, not a separate `alknet/docker` ALPN; the raw-carriage handoff the POC struggled with is dissolved by the alknet-tty extraction — interactive attach moved to `alknet/tty` via `DockerTtyBackend`, no `carriage` field on `call.requested`), [ADR-059](decisions/059-bollard-021-dependency-and-features.md) (bollard 0.21, verified current on crates.io; features `http`+`pipe`+`time`, no `ssl`/`ssh`/`websocket`/`buildkit` — single-host by construction, fleet is a call-protocol concern), [ADR-060](decisions/060-container-resource-model-and-label-namespace.md) (ADR-050 application to bollard: `alknet.managed`/`alknet.owner` labels; `list` `owned_only` flag; hosted-services operator role via the static-resource fallback; handler-driven `revoke` on `remove` with autonomous-death tolerance), [ADR-061](decisions/061-docker-tty-backend-in-alknet-docker.md) (`DockerTtyBackend` in alknet-docker behind a `tty` feature, not a sibling crate; attach vs exec mode; the POC's `drive_attach_raw` as the reference), [ADR-062](decisions/062-docker-client-injection-via-closure-capture.md) (the `Docker` client + `OwnershipStore` are closure-captured at registration time, not read from `OperationContext` and not smuggled through `Capabilities` — `Capabilities` is for secret material only per ADR-014; matches the `from_openapi` pattern), [ADR-063](decisions/063-exit-code-on-terminal-call-responded.md) (non-interactive exec puts `{ "exitCode": N, "terminal": true }` on a final `call.responded` before `call.completed` — `call.completed` stays empty, ADR-012 unchanged). The specs are grounded in the alknet-docker POC (`docs/research/alknet-docker/poc-summary.md`, `/workspace/alknet-docker-poc/`), which validated the hard parts (interactive attach, logs subscription, exec with exit code); the remaining lifecycle operations are mechanical bollard wrapping. The two use cases — disposable dev containers (coordinator-spawned, ownership-recorded) and long-running hosted services (operator-managed, static-resource fallback, per `/workspace/system/dev1/docker.md`) — both work through one `AccessControl` model (ADR-050/060). The `DockerTtyBackend` fills the `TtyBackend` row the alknet-tty spec left open. Four OQs (048–051) track deferred scope: network/volume ops, buildkit, system events subscription, and the full `CreateContainerOptions` surface (deferred to v1 implementation). +**Transport generalization sweep (2026-07-09).** Three commits landed a +clean sweep discovered when building an external app against the crates: +(1) the dead `irpc` / `irpc-derive` workspace deps were removed (no `.rs` +file ever imported irpc — the wire protocol is hand-rolled), recorded by +[ADR-064](decisions/064-irpc-never-integrated-hand-rolled-framing.md) +(supersedes ADR-005, which had accepted "irpc as the call protocol +foundation" based on the previous architecture but was never implemented +as stated); (2) the iroh dep migrated `0.35 → 1.0.2` (6 API surface edits, +no architectural change — unblocks `alknet-blobs`); (3) +[ADR-065](decisions/065-connection-from-stream-generic-single-stream.md) +adds `Connection::from_stream` / `from_bidi` — `Connection` now accepts any +`AsyncRead + AsyncWrite` pair, unblocking TCP+TLS, SSH channel dispatch, +WebTransport streams, and wasm streams through the same `HandlerRegistry` +as QUIC connections, with zero handler code changes. The +`MockConnection` / `ConnectionKind::Mock` test variants are removed (tests +use `from_stream` with `tokio::io::sink`/`empty`). See +[`docs/research/transport-generalization/findings.md`](../research/transport-generalization/findings.md) +for the full trace. + ## Architecture Documents | Document | Status | Description | @@ -33,13 +52,13 @@ The alknet-call crate is **implemented and reviewed** — both the server-side c | [overview.md](overview.md) | draft | Workspace-level overview, crate graph, shared types, design principles | | [open-questions.md](open-questions.md) | draft | OQ index — theme-grouped tables + Deferred/Blocked section; per-OQ files in [`questions/`](questions/) | | [crates/core/README.md](crates/core/README.md) | draft | alknet-core crate index | -| [crates/core/core-types.md](crates/core/core-types.md) | draft | ProtocolHandler, HandlerError, Connection, BiStream, StreamError | +| [crates/core/core-types.md](crates/core/core-types.md) | draft | ProtocolHandler, HandlerError, Connection (QUIC + `from_stream`), BiStream, StreamError | | [crates/core/endpoint.md](crates/core/endpoint.md) | draft | ALPN router, HandlerRegistry, accept loop, shutdown | | [crates/core/auth.md](crates/core/auth.md) | draft | AuthContext, Identity, IdentityProvider, AuthToken, resolution flow | | [crates/core/config.md](crates/core/config.md) | draft | StaticConfig, DynamicConfig, ArcSwap, ConfigReloadHandle | | [crates/call/README.md](crates/call/README.md) | draft | alknet-call crate index | -| [crates/call/call-protocol.md](crates/call/call-protocol.md) | draft | CallAdapter, EventEnvelope framing, stream model, PendingRequestMap, bidirectional calls, streaming subscribe example | -| [crates/call/operation-registry.md](crates/call/operation-registry.md) | draft | OperationSpec, Handler, OperationRegistry, AccessControl, capability injection, service discovery, irpc integration | +| [crates/call/call-protocol.md](crates/call/call-protocol.md) | draft | CallAdapter, hand-rolled EventEnvelope framing (no irpc — ADR-064), stream model, PendingRequestMap, bidirectional calls, streaming subscribe example | +| [crates/call/operation-registry.md](crates/call/operation-registry.md) | draft | OperationSpec, Handler, OperationRegistry, AccessControl, capability injection, service discovery (hand-rolled, no irpc) | | [crates/call/client-and-adapters.md](crates/call/client-and-adapters.md) | draft | CallClient (outbound connection opener), from_call / from_jsonschema, OperationAdapter trait, adapter location map, no-env-vars invariant, exchange-of-operations pattern | | [crates/http/README.md](crates/http/README.md) | draft | alknet-http crate index | | [crates/http/overview.md](crates/http/overview.md) | draft | Crate purpose, two roles (server + client host), dependencies, adapter location map | @@ -72,7 +91,7 @@ The alknet-call crate is **implemented and reviewed** — both the server-side c | [002](decisions/002-protocol-handler-trait.md) | ProtocolHandler Trait | Accepted | | [003](decisions/003-crate-decomposition.md) | Crate Decomposition | Accepted | | [004](decisions/004-auth-as-shared-core.md) | Auth as Shared Core (IdentityProvider) | Accepted | -| [005](decisions/005-irpc-as-call-protocol-foundation.md) | irpc as Call Protocol Foundation | Accepted | +| [005](decisions/005-irpc-as-call-protocol-foundation.md) | irpc as Call Protocol Foundation | ~~Accepted~~ → **Superseded** by ADR-064 (irpc was never integrated) | | [006](decisions/006-alpn-convention-and-connection-model.md) | ALPN String Convention and Connection Model | Accepted | | [007](decisions/007-bistream-type-definition.md) | BiStream Type Definition | Accepted | | [008](decisions/008-secret-service-integration.md) | Vault Integration Point | Accepted | @@ -131,6 +150,8 @@ The alknet-call crate is **implemented and reviewed** — both the server-side c | [061](decisions/061-docker-tty-backend-in-alknet-docker.md) | DockerTtyBackend in alknet-docker | Accepted | | [062](decisions/062-docker-client-injection-via-closure-capture.md) | Docker Client and OwnershipStore Injection via Closure Capture | Accepted | | [063](decisions/063-exit-code-on-terminal-call-responded.md) | Exit Code on a Terminal `call.responded` for Non-Interactive Exec | Accepted | +| [064](decisions/064-irpc-never-integrated-hand-rolled-framing.md) | irpc Was Never Integrated — Hand-Rolled EventEnvelope Framing | Accepted (supersedes ADR-005) | +| [065](decisions/065-connection-from-stream-generic-single-stream.md) | `Connection::from_stream` — Generic Single-Stream Connections | Accepted | ## Open Questions diff --git a/docs/architecture/crates/call/README.md b/docs/architecture/crates/call/README.md index 8d2a3dc..e39acc0 100644 --- a/docs/architecture/crates/call/README.md +++ b/docs/architecture/crates/call/README.md @@ -1,19 +1,19 @@ --- status: draft -last_updated: 2026-06-27 -review: call/review-call passed 2026-06-23 — registry, protocol, ADR (005/012/014/015/016/017/022/023/024), security, and pattern-consistency checks all conformant; 159 unit/integration tests green; `cargo build`, `cargo clippy -- -D warnings`, `cargo fmt --check`, `cargo test` clean. Call-completion gap (ADR-017 client/adapter surface) addressed 2026-06-26; ADR-029 migration pending. +last_updated: 2026-07-09 +review: call/review-call passed 2026-06-23 — registry, protocol, ADR (005/012/014/015/016/017/022/023/024), security, and pattern-consistency checks all conformant; 159 unit/integration tests green; `cargo build`, `cargo clippy -- -D warnings`, `cargo fmt --check`, `cargo test` clean. Call-completion gap (ADR-017 client/adapter surface) addressed 2026-06-26; ADR-029 migration pending. Transport generalization sweep (ADR-064 supersedes ADR-005; ADR-065 `from_stream`) synced 2026-07-09. --- # alknet-call -Structured RPC over QUIC: operations, request/response, streaming subscriptions, and service discovery. Implements `ProtocolHandler` on ALPN `alknet/call`. +Structured RPC: operations, request/response, streaming subscriptions, and service discovery. Implements `ProtocolHandler` on ALPN `alknet/call`. Runs over QUIC (quinn/iroh) and, via `Connection::from_stream` (ADR-065), over any `AsyncRead + AsyncWrite` transport. ## Documents | Document | Status | Description | |----------|--------|-------------| -| [call-protocol.md](call-protocol.md) | draft | CallAdapter, EventEnvelope framing, stream model, PendingRequestMap, bidirectional calls | -| [operation-registry.md](operation-registry.md) | draft | OperationSpec, Handler, OperationRegistry, AccessControl, service discovery, irpc integration | +| [call-protocol.md](call-protocol.md) | draft | CallAdapter, hand-rolled EventEnvelope framing (no irpc — ADR-064), stream model, PendingRequestMap, bidirectional calls | +| [operation-registry.md](operation-registry.md) | draft | OperationSpec, Handler, OperationRegistry, AccessControl, service discovery, hand-rolled framing (no irpc — ADR-064) | | [client-and-adapters.md](client-and-adapters.md) | draft | CallClient (outbound connection opener), from_call / from_jsonschema, OperationAdapter trait, adapter location map, no-env-vars invariant, exchange-of-operations pattern | ## Applicable ADRs @@ -22,10 +22,12 @@ Structured RPC over QUIC: operations, request/response, streaming subscriptions, |-----|-------|-----------| | [001](../../decisions/001-alpn-protocol-dispatch.md) | ALPN-Based Protocol Dispatch | CallAdapter registers on ALPN `alknet/call` | | [002](../../decisions/002-protocol-handler-trait.md) | ProtocolHandler Trait | CallAdapter implements ProtocolHandler | -| [003](../../decisions/003-crate-decomposition.md) | Crate Decomposition | alknet-call depends on alknet-core and irpc | +| [003](../../decisions/003-crate-decomposition.md) | Crate Decomposition | alknet-call depends on alknet-core (no irpc — ADR-064) | | [013](../../decisions/013-rust-canonical-implementation.md) | Rust as Canonical Implementation Language | Adapter traits defined in Rust; TS is reference/browser adaptation | | [004](../../decisions/004-auth-as-shared-core.md) | Auth as Shared Core | AuthContext passed to call handlers | -| [005](../../decisions/005-irpc-as-call-protocol-foundation.md) | irpc as Call Protocol Foundation | irpc provides framing and service dispatch | +| [005](../../decisions/005-irpc-as-call-protocol-foundation.md) | ~~irpc as Call Protocol Foundation~~ | ~~Accepted~~ → **Superseded** by [ADR-064](../../decisions/064-irpc-never-integrated-hand-rolled-framing.md) (irpc was never integrated; framing is hand-rolled) | +| [064](../../decisions/064-irpc-never-integrated-hand-rolled-framing.md) | Hand-Rolled EventEnvelope Framing | Wire format, registry, dispatch are hand-rolled in alknet-call; supersedes ADR-005 | +| [065](../../decisions/065-connection-from-stream-generic-single-stream.md) | `Connection::from_stream` | Generic single-stream connections; unblocks TCP+TLS/SSH/WT/wasm dispatch | | [006](../../decisions/006-alpn-convention-and-connection-model.md) | ALPN String Convention | `alknet/call` ALPN, one ALPN per connection | | [007](../../decisions/007-bistream-type-definition.md) | BiStream Type Definition | CallAdapter receives Connection, not BiStream | | [008](../../decisions/008-secret-service-integration.md) | Vault Integration Point | Vault accessed at assembly layer, not on the wire | @@ -72,7 +74,7 @@ Structured RPC over QUIC: operations, request/response, streaming subscriptions, 2. **Protocol is symmetric**: Both sides can initiate calls. The server calling a client uses the same EventEnvelope format and correlation. 3. **Stream-agnostic correlation**: PendingRequestMap correlates by request ID, not by stream. The protocol works with any stream arrangement. 4. **Operation registry is layered**: The curated layer (`Local` provenance) is static — registered at startup by the CLI binary, immutable for the process lifetime. Session (`Session`) and imported (`FromCall` etc.) ops are dynamic overlays at their respective scopes (per-session, per-connection). The registry supports JSON Schema discovery. See ADR-024. -5. **irpc is one dispatch backend**: Local operations dispatch directly. irpc service calls (in-process, type-safe) are internal. The call protocol is the external interface. +5. **Hand-rolled dispatch (no irpc)**: Operations dispatch through the hand-rolled `OperationRegistry` (ADR-064). The call protocol is the external interface; internal handler dispatch uses `Handler`/`StreamingHandler` trait objects (ADR-049), not an irpc service. 6. **Local dispatch only**: The operation registry dispatches to local handlers. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer, not a modification to alknet-call's path format. 7. **No secret material on the wire**: The call protocol carries no private keys, API keys, mnemonics, or decrypted credentials. Handlers receive outbound credentials through `OperationContext.capabilities`, injected at the assembly layer. See ADR-014. 8. **Abort cascades to descendants**: `call.aborted` for a parent request cascades to all non-terminal descendants. Default `abort-dependents`; `continue-running` opt-in. See ADR-016. diff --git a/docs/architecture/crates/call/call-protocol.md b/docs/architecture/crates/call/call-protocol.md index 4866c04..3ce5397 100644 --- a/docs/architecture/crates/call/call-protocol.md +++ b/docs/architecture/crates/call/call-protocol.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-07-02 +last_updated: 2026-07-09 --- # Call Protocol @@ -23,7 +23,10 @@ The protocol must be: - **Stream-agnostic**: QUIC provides stream multiplexing; the protocol shouldn't impose additional constraints - **Discoverable**: Clients can query what operations exist and their schemas -See ADR-005 for the decision to use irpc as the call protocol's foundation and ADR-012 for the stream model decision. +See ADR-064 for the decision that the call protocol uses hand-rolled +`EventEnvelope` framing (irpc was never integrated — ADR-005, which +accepted "irpc as the call protocol foundation," is superseded) and ADR-012 +for the stream model decision. ## Architecture @@ -210,7 +213,10 @@ The `Value` type is `serde_json::Value`. The envelope is JSON because it must be Binary payloads (postcard, protobuf) are base64-encoded as a JSON string within the `payload` field. The convention is: if an operation's output schema specifies a binary field, the handler encodes it as a base64 string and the client decodes it. The `EventEnvelope` structure is not aware of this convention — it carries a `serde_json::Value` and does not interpret the payload. This is a handler-level concern, not a protocol-level concern. -This is the same framing used by irpc. The Rust implementation in alknet-call is canonical — the `@alkdev/pubsub` TypeScript adapters serve as a reference and browser adaptation, not a parallel implementation (see ADR-013). +This is hand-rolled length-prefixed JSON framing (ADR-064), coincidentally +the same shape irpc uses. The Rust implementation in alknet-call is +canonical — the `@alkdev/pubsub` TypeScript adapters serve as a reference +and browser adaptation, not a parallel implementation (see ADR-013). ### Event Types @@ -546,7 +552,7 @@ Handlers clean up resources when their call is cancelled (in Rust, the future is - Operation specs use JSON Schema. The envelope is always JSON. Binary payloads may be base64-encoded in the `payload` field. - Batch is not a protocol primitive — multiple `call.requested` events with correlated IDs provide equivalent semantics. See OQ-14. - The call protocol is transport-agnostic at the envelope level. The `EventEnvelope` framing can run over QUIC streams, WebSocket frames, or Worker `postMessage`. The `CallAdapter` is the QUIC-specific implementation. **The `EventEnvelope` shape (`{ type, id, payload }`) was derived from the `@alkdev/pubsub` `EventEnvelope` (`/workspace/@alkdev/pubsub/src/types.ts`), which already has a working WebSocket client/server implementation (`event-target-websocket-client.ts` / `event-target-websocket-server.ts`) and a generalized "event target" abstraction. The call protocol refined the envelope with typed event names (`call.requested`, `call.responded`, etc.) and structured payloads; the delta is small and well-defined, making a browser (and Node) WebSocket client straightforward to derive from the pubsub prior art. See ADR-044, [ADR-048](../../decisions/048-websocket-native-session-not-gateway.md), and [websocket.md](../http/websocket.md). -- `OperationEnv::invoke()` dispatches through the local registry. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer. See ADR-005 and OQ-13. +- `OperationEnv::invoke()` dispatches through the local registry. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer. See ADR-064 and OQ-13. - **The call protocol carries no secret material.** Secret material (private keys, API keys, mnemonics, decrypted credentials, raw tokens) must not appear in `call.requested` payloads, `call.responded` payloads, or `OperationContext.metadata`. The wire format carries `serde_json::Value` and cannot enforce this at the type level — the constraint is architectural, enforced by the operation registry and by convention. Operations that need to share public key material use a dedicated operation that returns only the public component. See ADR-014. - **Abort cascades to descendants.** `call.aborted` for a parent request cascades to all non-terminal descendants in the call tree. Default policy is `abort-dependents`; `continue-running` is an opt-in. See ADR-016. @@ -554,7 +560,7 @@ Handlers clean up resources when their call is cancelled (in Rust, the future is | Decision | ADR | Summary | |----------|-----|---------| -| irpc as call protocol foundation | [ADR-005](../../decisions/005-irpc-as-call-protocol-foundation.md) | irpc provides framing and service dispatch | +| Hand-rolled EventEnvelope framing (irpc never integrated) | [ADR-064](../../decisions/064-irpc-never-integrated-hand-rolled-framing.md) | Hand-rolled length-prefixed JSON framing, operation registry, dispatch; supersedes ADR-005 (irpc was never imported) | | Call protocol stream model | [ADR-012](../../decisions/012-call-protocol-stream-model.md) | Bidirectional streams, EventEnvelope, ID-based correlation | | ALPN per connection | [ADR-006](../../decisions/006-alpn-convention-and-connection-model.md) | `alknet/call` is a distinct ALPN, one connection per ALPN | | ProtocolHandler receives Connection | [ADR-007](../../decisions/007-bistream-type-definition.md) | CallAdapter gets Connection, can accept/open multiple streams | @@ -615,7 +621,7 @@ See [open-questions.md](../../open-questions.md) for full details. - [operation-registry.md](operation-registry.md) — OperationSpec, Handler, AccessControl, service discovery - [client-and-adapters.md](client-and-adapters.md) — CallClient, from_call, OperationAdapter, peer-keyed composition env -- ADR-005: irpc as call protocol foundation +- ADR-064: Hand-rolled EventEnvelope framing (irpc never integrated; supersedes ADR-005) - ADR-012: Call protocol stream model - ADR-029: Peer-graph routing model (peer-keyed overlays + `PeerRef` routing) - ADR-030: PeerEntry and Identity.id decoupling (`PeerId` source) diff --git a/docs/architecture/crates/call/operation-registry.md b/docs/architecture/crates/call/operation-registry.md index 53ae95b..1ba56eb 100644 --- a/docs/architecture/crates/call/operation-registry.md +++ b/docs/architecture/crates/call/operation-registry.md @@ -5,7 +5,7 @@ last_updated: 2026-07-05 # Operation Registry -OperationSpec, Handler, OperationRegistry, AccessControl, service discovery, and irpc integration. +OperationSpec, Handler, OperationRegistry, AccessControl, service discovery, and the hand-rolled framing (no irpc — ADR-064). ## What @@ -185,7 +185,11 @@ pub type ResponseStream = Pin + Send>>; ``` Both handlers are async — many operations (file I/O, HTTP service calls, -irpc service calls, LLM streaming) are inherently asynchronous. A handler +LLM streaming) are inherently asynchronous. A handler (whether it wraps a +local function, an HTTP-backed OpenAPI operation, an LLM stream, or a +`from_call` remote) is a `Future` (for `Query`/`Mutation`) or a `Stream` +(for `Subscription`, ADR-049). The registry's `Handler` / +`StreamingHandler` trait objects (ADR-049) abstract over this. A handler receives: - `input: Value` — the deserialized `payload` from the `call.requested` event @@ -447,7 +451,7 @@ The CLI binary (or assembly layer) constructs the registry and passes it to the ### OperationEnv -The `OperationEnv` trait is the universal composition mechanism. A handler calls `context.env.invoke("fs", "readFile", input, &context)` and gets a `ResponseEnvelope` back — regardless of whether the operation runs locally, via an irpc service, or on a remote node. +The `OperationEnv` trait is the universal composition mechanism. A handler calls `context.env.invoke("fs", "readFile", input, &context)` and gets a `ResponseEnvelope` back — regardless of whether the operation runs locally or on a remote node. **`OperationEnv` is request/response-only** (ADR-049). It returns a single `ResponseEnvelope` — no streaming variant exists. Calling `invoke()` on a `Subscription` op produces `CallError { code: "INVALID_OPERATION_TYPE", ... }` — composition cannot truncate a stream to its first value. Stream composition (filter, map, combine, window, dedupe) is a handler-level concern, not a protocol composition concern; see ADR-049 for the rationale and the `@alkdev/pubsub` `operators.ts` prior art. @@ -736,7 +740,7 @@ Two things happen in `invoke()`: 1. **Reachability check**: before constructing the child context, `invoke()` checks whether the requested op is in the parent's scoped env. If not, `NOT_FOUND`. This is the reachability control — a handler can only compose declared ops. 2. **Authority propagation**: the child's `identity` is the parent's `handler_identity` (the parent's composition authority becomes the caller). The child's `handler_identity` is the child's own registration's `composition_authority` — so if the child itself composes further, its children inherit the child's authority. This is the principal/agent chain from ADR-015, now wired via ADR-022. -Future work may add irpc service dispatch and remote call protocol dispatch as additional backends. The handler-facing API stays the same. +Future work may add remote call protocol dispatch as an additional backend. The handler-facing API stays the same. **`OperationEnv` must remain a trait.** This is a constraint, not a suggestion. The trait-based design enables registry layering (ADR-024): the CallAdapter composes the root env per call from the curated base + active peer-keyed connection overlays + session overlay, and overlays wrap the base via trait layering. Session-scoped registries (OQ-19) and connection-scoped remote imports (ADR-017 `from_call`) are both overlays on the same base, using the same mechanism. The peer-keyed extension (`PeerCompositeEnv`, `invoke_peer`, ADR-029) composes on top of the same trait — it overrides the new peer-routing methods, not the base dispatch. Making `OperationEnv` concrete or hardcoding the global registry into the dispatch path would close both the session-overlay and connection-overlay patterns, and would prevent the peer-keyed routing model from composing. This is the same integration-point pattern as `IdentityProvider` (ADR-004). See OQ-19, ADR-024, and ADR-029. @@ -774,18 +778,21 @@ from wire `operationId`s before lookup, so `services/schema` accepts both client reading the schema can produce typed error enums instead of generic error handling. -### irpc Integration +### Operation Registry (hand-rolled, no irpc) -irpc and the operation registry serve different scopes: +The operation registry is hand-rolled in alknet-call. ADR-005 accepted +"irpc as the call protocol foundation," but no `.rs` file in the workspace +ever imported irpc — the wire format (`wire.rs`), the operation registry, +and the dispatch are all hand-rolled. ADR-064 supersedes ADR-005 and +records the actual state. The table that previously contrasted "call +protocol (external, JSON)" with "irpc services (internal, postcard)" is +moot — there is no irpc layer. -| Layer | Mechanism | Serialization | Scope | -|-------|-----------|---------------|-------| -| Call protocol (external) | `EventEnvelope` over QUIC streams | JSON | Cross-language, cross-node | -| irpc services (internal) | `#[rpc_requests]` derive macro, `Service` trait | postcard (binary) | Rust-to-Rust, in-process or in-cluster | - -irpc services are an internal dispatch mechanism — they are not directly exposed on the call protocol. alknet-call itself uses irpc for its call-protocol framing (ADR-005); the vault no longer uses irpc (ADR-025 — direct method calls on `VaultServiceHandle`). The vault is accessed by the assembly layer (CLI binary) at startup, not by handlers at call time. See ADR-008 and ADR-014. - -If a handler internally uses an irpc-based service, the handler bridges the two: it receives JSON input from the call protocol, calls the irpc service in-process (postcard, type-safe), and serializes the result back to JSON for the call protocol response. This layering preserves irpc's type safety for internal calls while keeping the external interface cross-language. +If a handler internally uses a postcard/binary RPC for in-process calls, +that's a handler-internal choice, not an alknet-call integration. The +operation registry's external interface is always JSON (the `EventEnvelope` +wire format); the internal handler dispatch is a `Handler` / +`StreamingHandler` trait object (ADR-049), not an irpc `Service`. ### Operation Registration at Startup @@ -874,8 +881,8 @@ The `Capabilities` type holds non-serializable, zeroized secret material. It doe ## Constraints - The registry is **layered by trust boundary** (ADR-024). The curated layer (`Local` provenance) is immutable after construction — adding a `Local` op requires restarting the process, which re-enters the startup trust boundary. Session (`Session`) and imported (`FromCall` etc.) ops are dynamic at their respective scopes (per-session, per-connection). The pre-ADR-024 blanket immutability claim was inherited by analogy from ADR-010's `HandlerRegistry` (ALPN-level) and did not apply to the operation registry — the TLS-config argument that justifies `HandlerRegistry` immutability does not touch the operation registry, which lives behind the single ALPN `alknet/call`. -- Operation specs use JSON Schema. The call protocol's external interface is always JSON. irpc's postcard serialization is internal only. -- `OperationEnv::invoke()` dispatches through the local registry. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer — not a prefix added to operation paths. irpc service dispatch is contracted but not built. +- Operation specs use JSON Schema. The call protocol's external interface is always JSON. Internal handler dispatch is via `Handler` / `StreamingHandler` trait objects (ADR-049), not a binary RPC framework. +- `OperationEnv::invoke()` dispatches through the local registry. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer — not a prefix added to operation paths. - The call protocol does not depend on any database. Operation specs are in-memory, populated at startup. - `OperationContext.internal` is set by `OperationEnv`, not by callers. A handler cannot mark its own call as internal. The `internal` flag switches authority context (composition authority for ACL), it does not skip ACL — see ADR-015, ADR-022. - **Operations have External/Internal visibility.** `Internal` operations return `NOT_FOUND` when called from the wire and are excluded from `services/list`. The assembly layer declares visibility at registration. See ADR-015. @@ -890,7 +897,7 @@ The `Capabilities` type holds non-serializable, zeroized secret material. It doe | Decision | ADR | Summary | |----------|-----|---------| -| irpc as call protocol foundation | [ADR-005](../../decisions/005-irpc-as-call-protocol-foundation.md) | irpc provides framing and service dispatch | +| Hand-rolled EventEnvelope framing (irpc never integrated) | [ADR-064](../../decisions/064-irpc-never-integrated-hand-rolled-framing.md) | Hand-rolled framing, registry, dispatch; supersedes ADR-005 | | Call protocol stream model | [ADR-012](../../decisions/012-call-protocol-stream-model.md) | Bidirectional streams, EventEnvelope, ID-based correlation | | Static handler registration | [ADR-010](../../decisions/010-alpn-router-and-endpoint.md) | `HandlerRegistry` (ALPN-level) immutable after construction; `OperationRegistry` layered by ADR-024 (curated immutable, session/imported dynamic) | | Vault integration via assembly layer | [ADR-008](../../decisions/008-secret-service-integration.md) | Vault is a capability source, accessed at assembly time | @@ -949,7 +956,7 @@ See [open-questions.md](../../open-questions.md) for full details. ## References - [call-protocol.md](call-protocol.md) — CallAdapter, EventEnvelope, stream model, PendingRequestMap -- ADR-005: irpc as call protocol foundation +- ADR-064: Hand-rolled EventEnvelope framing (irpc never integrated; supersedes ADR-005) - ADR-008: Vault integration point - ADR-010: ALPN router and endpoint (static registration — applies to the `HandlerRegistry`, not the `OperationRegistry`; see ADR-024 for the distinction) - ADR-012: Call protocol stream model diff --git a/docs/architecture/crates/core/core-types.md b/docs/architecture/crates/core/core-types.md index c5a023c..46e0333 100644 --- a/docs/architecture/crates/core/core-types.md +++ b/docs/architecture/crates/core/core-types.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-06-23 +last_updated: 2026-07-09 --- # Core Types @@ -47,11 +47,15 @@ Handler panics are caught by tokio's task isolation. The connection is dropped, ## Connection -An opaque type wrapping a QUIC connection. Handlers receive a `Connection` in `handle()`. +An opaque type wrapping a transport connection. Handlers receive a +`Connection` in `handle()`. The connection may be QUIC (quinn or iroh) or a +generic single stream (TCP+TLS, SSH channel, WebTransport stream, wasm +stream) — see ADR-065 for the `from_stream` generalization. ```rust pub struct Connection { - // Private: wraps the underlying QUIC connection or test mock + // Private: wraps the underlying connection — QUIC (quinn/iroh) or a + // generic single-stream pair (ConnectionKind::Stream). // Private: handler-resolved identity for observability (OQ-11) identity: OnceLock, } @@ -65,6 +69,24 @@ impl Connection { #[cfg(feature = "iroh")] pub fn from_iroh(conn: iroh::Connection) -> Self; + /// Construct from any pre-split read/write pair. `accept_bi()` yields + /// this pair once, then returns `ConnectionClosed`. `open_bi()` returns + /// `StreamClosed`. No feature gate — generic, no transport deps. + pub fn from_stream( + send: impl AsyncWrite + Send + Unpin + 'static, + recv: impl AsyncRead + Send + Unpin + 'static, + alpn: Vec, + remote_addr: Option, + ) -> Self; + + /// Convenience for a single bidirectional stream (e.g. + /// `TlsStream`). Splits internally via `tokio::io::split`. + pub fn from_bidi( + stream: impl AsyncRead + AsyncWrite + Send + Unpin + 'static, + alpn: Vec, + remote_addr: Option, + ) -> Self; + pub async fn accept_bi(&self) -> Result<(SendStream, RecvStream), StreamError>; pub async fn open_bi(&self) -> Result<(SendStream, RecvStream), StreamError>; pub fn remote_alpn(&self) -> &[u8]; @@ -75,14 +97,32 @@ impl Connection { } ``` -- `accept_bi()`: Wait for the peer to open a bidirectional stream. Returns `(SendStream, RecvStream)`. -- `open_bi()`: Open a bidirectional stream to the peer. Returns `(SendStream, RecvStream)`. +- `accept_bi()`: Yield the next bidirectional stream this connection + provides. **Transport semantics (ADR-065):** QUIC (quinn/iroh) returns a + new bidi stream on each call, `ConnectionClosed` when the underlying + connection closes; a single-stream connection (TCP+TLS, SSH channel, + WebTransport stream, wasm stream) yields the underlying stream on the + first call, then `ConnectionClosed` on all subsequent calls. Handlers + that loop `accept_bi` (TtyAdapter) get one session per single-stream + connection; handlers that call once (HttpAdapter) get the stream + directly. Both correct, no branching on transport. +- `open_bi()`: Open a bidirectional stream to the peer. Returns + `(SendStream, RecvStream)`. On a single-stream connection, returns + `StreamClosed` — a single stream cannot open new application streams. - `remote_alpn()`: The ALPN negotiated for this connection. Always present. - `remote_addr()`: The peer's address, if available. Informational (NAT/proxy). -- `close()`: Close the connection with an error code and reason. +- `close()`: Close the connection with an error code and reason. The + `code`/`reason` args are QUIC-specific (application-level close codes); + for a raw stream they're ignored — the drop is the close. - `set_identity()`: Store the handler-resolved identity for observability (OQ-11). Write-once-read-many — a second call returns an error. Handlers that resolve identity inside `handle()` call this; the identity is read by handler-side logging (the handler logs which identity it resolved) and is available on the `Connection` for any code that holds a reference to it. The endpoint does **not** read `identity()` after `handle()` returns — the `Connection` is moved into the spawned handler task (endpoint.md), so the endpoint no longer has a reference. Connection-level observability (remote addr, ALPN, connection ID) is logged by the endpoint before the move; identity-level observability is logged by the handler. See OQ-11 for the full resolution. -The `Connection` type does not expose quinn types in its public API. It wraps `quinn::Connection` internally, but the wrapper allows test implementations. +The `Connection` type does not expose quinn/iroh types in its public API. +It wraps the underlying connection internally via `ConnectionKind` enum +dispatch (`Quinn` / `Iroh` / `Stream`), with the QUIC variants feature-gated +and the `Stream` variant always available (no transport deps). See +[ADR-065](../../decisions/065-connection-from-stream-generic-single-stream.md) +for the `from_stream` generalization and the yield-once `accept_bi` +contract. See [ADR-007](../../decisions/007-bistream-type-definition.md) for why handlers receive Connection instead of BiStream. @@ -96,18 +136,21 @@ pub trait BiStream: AsyncRead + AsyncWrite + Send + Unpin {} Handlers that only need a single stream can obtain one via `connection.accept_bi()` and treat the `(SendStream, RecvStream)` pair as a BiStream. The `BiStream` trait is a convenience for: - Client-side code that has a single bidirectional stream -- Test mocks that need to simulate a stream -- Future transport abstractions (WebTransport, raw TCP) that produce bidirectional byte streams +- Test scenarios that need to simulate a stream +- Transports that produce a single bidirectional byte stream (TCP+TLS via `from_bidi`, SSH channels, WebTransport streams, wasm streams) — all dispatchable through the same `HandlerRegistry` as QUIC connections via `Connection::from_stream` (ADR-065) See [ADR-007](../../decisions/007-bistream-type-definition.md) for why BiStream is a trait. ## SendStream and RecvStream -Concrete types wrapping QUIC stream halves. Both quinn and iroh produce QUIC connections — `SendStream` and `RecvStream` need to wrap either source. +Concrete types wrapping transport stream halves. Both quinn and iroh +produce QUIC connections; `from_stream` adds a generic single-stream source. +`SendStream` and `RecvStream` wrap any of the three via internal enum +dispatch. ```rust -pub struct SendStream { /* wraps quinn::SendStream or iroh::SendStream or test mock */ } -pub struct RecvStream { /* wraps quinn::RecvStream or iroh::RecvStream or test mock */ } +pub struct SendStream { /* wraps quinn::SendStream, iroh::SendStream, or a generic Box */ } +pub struct RecvStream { /* wraps quinn::RecvStream, iroh::RecvStream, or a generic Box */ } impl AsyncWrite for SendStream { ... } impl AsyncRead for RecvStream { ... } @@ -115,9 +158,18 @@ impl AsyncRead for RecvStream { ... } - `SendStream` implements `AsyncWrite`. Write bytes to the peer. - `RecvStream` implements `AsyncRead`. Read bytes from the peer. -- These are concrete wrapper types that use internal enum dispatch to delegate to the appropriate QUIC stream type (quinn or iroh) in production, and to test mocks in tests. +- These are concrete wrapper types that use internal enum dispatch to + delegate to the appropriate stream source: quinn or iroh (QUIC, + feature-gated) in production, or `Stream` (a generic + `Box`, no feature gate) for + single-stream connections constructed via `from_stream` / `from_bidi`. -Since the endpoint supports both quinn and iroh connection sources (ADR-010), streams may come from either. `Connection::from_quinn()` / `Connection::from_iroh()` wrap the appropriate stream source based on where the connection came from. +Since the endpoint supports both quinn and iroh connection sources +(ADR-010), and `from_stream` adds the generic single-stream source +(ADR-065), streams may come from any of the three. `Connection::from_quinn()` +/ `from_iroh()` wrap the appropriate QUIC stream source based on where the +connection came from; `Connection::from_stream()` / `from_bidi()` wrap a +generic `AsyncRead + AsyncWrite` pair as the `Stream` variant. ## StreamError @@ -145,6 +197,8 @@ When a handler encounters a `StreamError` and needs to return from `handle()`, i Handlers that manage multiple streams (SSH, call) may catch `StreamError::StreamClosed` per-stream and continue serving other streams on the same connection — only `ConnectionClosed` forces `handle()` to return. +**Note on single-stream connections (ADR-065):** `StreamClosed` from `open_bi` on a `ConnectionKind::Stream` (single-stream) connection is terminal for that connection — no other streams exist to continue with. The "connection may still be usable" framing above applies to the QUIC case (a per-stream closure where the connection lives); the single-stream case is a transport property (one stream is all there is), not a mid-operation stream closure. `accept_bi` on a single-stream connection returns `ConnectionClosed` after the first yield (not `StreamClosed`), so handlers that loop `accept_bi` exit cleanly. + The mapping is provided as a `From` impl so handlers can use the `?` operator: ```rust @@ -245,8 +299,9 @@ registration bundle. |----------|-----|---------| | ProtocolHandler receives Connection, not BiStream | [ADR-007](../../decisions/007-bistream-type-definition.md) | Handlers that need multiple streams (SSH, call) have direct access to the Connection | | BiStream is a trait | [ADR-007](../../decisions/007-bistream-type-definition.md) | WASM door preserved, test mocks possible | +| `Connection::from_stream` — generic single-stream connections | [ADR-065](../../decisions/065-connection-from-stream-generic-single-stream.md) | `from_stream`/`from_bidi` accept any `AsyncRead + AsyncWrite`; yield-once `accept_bi` contract; unblocks TCP+TLS, SSH channels, WebTransport, wasm; QUIC variants feature-gated, `Stream` variant always available; `MockConnection`/`ConnectionKind::Mock` removed (tests use `from_stream` with `sink`/`empty`) | | HandlerError is non-fatal | [ADR-010](../../decisions/010-alpn-router-and-endpoint.md) | Handler errors close the connection, not the endpoint | -| SendStream/RecvStream wrap quinn + iroh | [ADR-010](../../decisions/010-alpn-router-and-endpoint.md) | Internal enum dispatch for both QUIC sources | +| SendStream/RecvStream wrap quinn + iroh + generic streams | [ADR-010](../../decisions/010-alpn-router-and-endpoint.md), [ADR-065](../../decisions/065-connection-from-stream-generic-single-stream.md) | Internal enum dispatch for QUIC sources and the generic `Stream` variant | | Connection stores handler-resolved identity | OQ-11 (resolved) | `set_identity` via `OnceLock` — write-once-read-many; read by handler-side logging, not by the endpoint (C13 resolved) | | Capabilities type | [ADR-014](../../decisions/014-secret-material-flow-and-capability-injection.md) | Non-serializable, zeroized, immutable after construction; `Clone` for composition propagation | diff --git a/docs/architecture/crates/core/endpoint.md b/docs/architecture/crates/core/endpoint.md index 7542523..1de8b0e 100644 --- a/docs/architecture/crates/core/endpoint.md +++ b/docs/architecture/crates/core/endpoint.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-06-22-17 +last_updated: 2026-07-09 --- # Endpoint @@ -37,9 +37,27 @@ A node can be reachable through different paths depending on its network context These are not interchangeable transports — they are **complementary connectivity modes**. A node behind NAT that also has a public IP can use both simultaneously. Both produce QUIC connections that dispatch through the same `HandlerRegistry` by ALPN string. -### TCP is NOT an endpoint concern +### TCP is NOT an endpoint struct concern (but CAN dispatch through the registry) -Bare TCP (SSH over port 22) does not use QUIC or ALPN. In the new model, TCP access is handled by individual handlers — the SSH handler can listen on a TCP socket independently. This is a handler-specific concern, not a core endpoint concern. +Bare TCP (SSH over port 22) does not use QUIC or ALPN. TCP access is not +owned by the `AlknetEndpoint` struct — there is no `tcp: +Option` field. The endpoint manages QUIC connection sources +(quinn + iroh) only. + +This does **not** mean TCP+TLS can't participate in ALPN dispatch. Since +[ADR-065](../../decisions/065-connection-from-stream-generic-single-stream.md), +`Connection::from_bidi(tls_stream, alpn, remote_addr)` constructs a +`Connection` from any `TlsStream` (or any `AsyncRead + +AsyncWrite` pair). A TCP+TLS accept loop built *outside* the endpoint (by +the assembly layer or a handler) can wrap each TLS stream as a +`Connection` and dispatch through the **same `HandlerRegistry`** the +endpoint uses, by the ALPN negotiated in the TLS handshake. This is not a +parallel listener bypassing the core — it's the same ALPN dispatch, over +a non-QUIC transport. `HttpAdapter`, `TtyAdapter`, and the call handler +all work over the single stream unchanged (ADR-065's yield-once +`accept_bi` contract). The TCP+TLS accept loop itself is a follow-up +commit, not part of `AlknetEndpoint`; the primitive it needs +(`from_bidi`) is in place. The reference implementation's TCP transport (`alknet-main/crates/alknet-core/src/transport/tcp.rs`) is SSH-specific. It doesn't generalize to the ALPN model. @@ -228,7 +246,10 @@ Note: `TlsIdentity::RawKey` uses `Ed25519SecretKey` (alknet-core-owned, backed by `ed25519-dalek`), not `iroh::SecretKey`. It is available in quinn-only builds without the `iroh` feature. When the iroh transport is also configured, `build_iroh_endpoint` converts the key to -`iroh::SecretKey::from_bytes` (ADR-027). +`iroh::SecretKey::from_bytes` (ADR-027). The iroh dep is on `1.0` +(`default-features = false, features = ["tls-aws-lc-rs"]`, matching the +quinn path's aws-lc-rs crypto provider); migrated from `0.35` in commit +`acd049e` (2026-07-09) — 6 API surface edits, no architectural change. ## Graceful Shutdown @@ -289,7 +310,7 @@ Non-fatal errors within a handler. See [core-types.md](core-types.md) for detail |----------|-----|---------| | Multi-connectivity endpoint (quinn + iroh) | [ADR-010](../../decisions/010-alpn-router-and-endpoint.md) | Both optional, both feed same ALPN router | | Static handler registration | [ADR-010](../../decisions/010-alpn-router-and-endpoint.md) | Two-way door, start static, add ArcSwap later | -| TCP is not an endpoint concern | [ADR-010](../../decisions/010-alpn-router-and-endpoint.md) | TCP SSH is a handler concern, not core | +| TCP is not an endpoint struct concern (but dispatches via `from_stream`) | [ADR-010](../../decisions/010-alpn-router-and-endpoint.md), [ADR-065](../../decisions/065-connection-from-stream-generic-single-stream.md) | `AlknetEndpoint` is QUIC-only (no `tcp` field); a TCP+TLS loop outside the endpoint wraps streams via `from_bidi` and shares the registry | | No byte-peeking, ALPN dispatch only | [ADR-001](../../decisions/001-alpn-protocol-dispatch.md) | TLS layer handles protocol detection | | Stealth mode = HTTP handler on standard ALPNs | [ADR-010](../../decisions/010-alpn-router-and-endpoint.md) | Decoy via ALPN routing, not byte-peek | | Network identity ≠ auth identity | [ADR-010](../../decisions/010-alpn-router-and-endpoint.md) | TLS cert/NodeId = network, SSH key/token = auth | diff --git a/docs/architecture/crates/http/http-server.md b/docs/architecture/crates/http/http-server.md index 06fcb6a..8adfb08 100644 --- a/docs/architecture/crates/http/http-server.md +++ b/docs/architecture/crates/http/http-server.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-07-02 +last_updated: 2026-07-09 --- # HTTP Server @@ -84,12 +84,18 @@ that do know about alknet. ## Architecture -### Running axum over a QUIC stream +### Running axum over a bidirectional stream The `HttpAdapter::handle()` method for `h2`/`http/1.1`: -1. Accepts one bidirectional stream from the QUIC connection - (`connection.accept_bi()` → `(SendStream, RecvStream)`). +1. Accepts one bidirectional stream from the connection + (`connection.accept_bi()` → `(SendStream, RecvStream)`). Over QUIC + this is one of many streams the connection provides; over a + single-stream connection (TCP+TLS via `Connection::from_bidi`, + ADR-065) it is the one stream, yielded once then `ConnectionClosed`. + Either way, `accept_bi` returns the `(SendStream, RecvStream)` pair + the adapter needs — the handler code is transport-agnostic (ADR-065's + yield-once contract). 2. Wraps the `(SendStream, RecvStream)` pair as a hyper `TokioIo`-compatible duplex stream — the same byte stream hyper expects for an HTTP connection. diff --git a/docs/architecture/crates/http/webtransport.md b/docs/architecture/crates/http/webtransport.md index 8eee9fa..48dcf6d 100644 --- a/docs/architecture/crates/http/webtransport.md +++ b/docs/architecture/crates/http/webtransport.md @@ -259,9 +259,14 @@ protocol (SSH, SFTP, git) reads/writes the WebTransport stream as a `BiStream` (ADR-007). The `BiStream` trait (`AsyncRead + AsyncWrite + Send + Unpin`) was designed for this — a browser implements it over a WebTransport stream, and the WASM parser speaks the protocol over it. -The WASM parsers are downstream artifacts (the SSH WASM client, the -SFTP WASM client), not part of `alknet-http`; `russh-sftp`'s WASM -targeting demonstrates feasibility, SSH is the next target. +On the server side, the `h3` handler wraps each WebTransport stream as a +`Connection` via `Connection::from_stream` (ADR-065) before handing it to +the target ALPN handler — the target handler runs its normal protocol over +the stream, unchanged from its QUIC path (the yield-once `accept_bi` +contract makes a single WebTransport stream look like a one-stream +connection). The WASM parsers are downstream artifacts (the SSH WASM +client, the SFTP WASM client), not part of `alknet-http`; `russh-sftp`'s +WASM targeting demonstrates feasibility, SSH is the next target. **Auth for proxied ALPN sessions:** the browser authenticates by bearer token on the WebTransport session request (the HTTP `Authorization` diff --git a/docs/architecture/decisions/003-crate-decomposition.md b/docs/architecture/decisions/003-crate-decomposition.md index a79646d..401b51b 100644 --- a/docs/architecture/decisions/003-crate-decomposition.md +++ b/docs/architecture/decisions/003-crate-decomposition.md @@ -24,10 +24,10 @@ The workspace decomposes into the following crates: | Crate | Responsibility | Depends on | |-------|---------------|------------| -| `alknet-core` | ProtocolHandler trait, ALPN router, endpoint, BiStream, AuthContext, IdentityProvider, config, ArcSwap dynamic config | tokio, quinn, rustls, irpc, iroh (feature-gated, added by ADR-010) | +| `alknet-core` | ProtocolHandler trait, ALPN router, endpoint, BiStream, AuthContext, IdentityProvider, config, ArcSwap dynamic config | tokio, quinn, rustls, iroh (feature-gated, added by ADR-010) | | `alknet-vault` | Local key vault: BIP39/SLIP-0010/AES-GCM key derivation, encryption | (standalone, no alknet-core) | | `alknet-ssh` | SshAdapter (russh, SOCKS5, port forwarding) | alknet-core, russh | -| `alknet-call` | CallAdapter (JSON-RPC via irpc, operation registry, pub/sub, access control, call protocol client, adapter traits) | alknet-core, irpc | +| `alknet-call` | CallAdapter (JSON-RPC via hand-rolled EventEnvelope framing, operation registry, pub/sub, access control, call protocol client, adapter traits) | alknet-core | | `alknet-agent` | Agent service: LLM execution loop (forked aisdk), tool dispatch via call protocol, provider key retrieval via vault | alknet-call | | `alknet-git` | GitAdapter (gix, pkt-line protocol) | alknet-core, gix | | `alknet-sftp` | SftpAdapter (russh-sftp protocol core) | alknet-core, russh-sftp | @@ -72,7 +72,7 @@ alknet-napi is a thin projection layer — it exposes the Rust call protocol cli - ADR-001: ALPN-based protocol dispatch - ADR-002: ProtocolHandler trait - ADR-004: Auth as shared core (IdentityProvider) -- ADR-005: irpc as call protocol foundation +- ADR-005: irpc as call protocol foundation (superseded by ADR-064) ## Amendments @@ -125,4 +125,19 @@ exception remains for alknet-http/agent/napi (which use alknet-call's not framing glue); it no longer covers alknet-tty. See [ADR-057](057-alknet-tty-no-alknet-call-dep.md) for the full decision and the three options considered (duplicate / promote to core / use -alknet-call). \ No newline at end of file +alknet-call). + +### Amendment 3 (2026-07-09): irpc is not a dependency of any crate + +The Decision table listed `irpc` as a dependency of `alknet-core` ("tokio, +quinn, rustls, irpc, iroh") and `alknet-call` ("alknet-core, irpc"). This +was carried over from the previous architecture and never verified against +the implementation: **no `.rs` file in the workspace ever imported irpc**. +The call protocol's wire format (`crates/alknet-call/src/protocol/wire.rs`) +is hand-rolled length-prefixed JSON; the `EventEnvelope` shape was derived +from the `@alkdev/pubsub` TypeScript prior art (ADR-013), not from irpc. +The dead `irpc` / `irpc-derive` workspace deps and the `alknet-call` consumer +dep were removed in commit `668d777`. See +[ADR-064](064-irpc-never-integrated-hand-rolled-framing.md) for the full +record (ADR-005, which accepted "irpc as the call protocol foundation," is +superseded). \ No newline at end of file diff --git a/docs/architecture/decisions/005-irpc-as-call-protocol-foundation.md b/docs/architecture/decisions/005-irpc-as-call-protocol-foundation.md index fe4be98..fc61b73 100644 --- a/docs/architecture/decisions/005-irpc-as-call-protocol-foundation.md +++ b/docs/architecture/decisions/005-irpc-as-call-protocol-foundation.md @@ -2,7 +2,22 @@ ## Status -Accepted +~~Accepted~~ → **Superseded** by [ADR-064](064-irpc-never-integrated-hand-rolled-framing.md) + +> **Superseded 2026-07-09.** This ADR accepted "irpc as the call protocol +> foundation" based on the previous architecture's use of irpc. When the +> call protocol was implemented, it turned out that **no `.rs` file in the +> workspace ever imported irpc** — the `irpc` / `irpc-derive` workspace deps +> were a Cargo.toml entry with no corresponding import. The wire protocol +> (`crates/alknet-call/src/protocol/wire.rs`) is hand-rolled length-prefixed +> JSON; the `EventEnvelope` shape was derived from the `@alkdev/pubsub` +> TypeScript prior art (ADR-013), not from irpc. ADR-064 supersedes this +> ADR and records the actual state: hand-rolled framing, no irpc +> integration. The architectural properties this ADR sought (proven +> length-prefixed JSON framing, cross-language JSON wire format, streaming) +> are preserved by the hand-rolled implementation. The text below is kept +> as the historical record of the decision that was made (and never +> implemented as stated). ## Context @@ -53,8 +68,11 @@ local-only by construction, not remote-capable by default). ## References +- **Superseding ADR**: [ADR-064](064-irpc-never-integrated-hand-rolled-framing.md) — irpc was never integrated; hand-rolled framing is the actual state +- ADR-013: Rust as canonical implementation (the `@alkdev/pubsub` prior art the `EventEnvelope` shape was actually derived from) +- ADR-025: Vault local-only dispatch (dropped irpc from the vault; ADR-064 confirms irpc was never in alknet-call either) - Pivot proposal: `docs/research/pivot/alpn-service-architecture.md` - ADR-003: Crate decomposition - ADR-004: Auth as shared core (IdentityProvider) -- irpc reference: `docs/research/references/iroh/irpc/` (see individual docs in that directory) +- Call protocol wire format (actual): `crates/alknet-call/src/protocol/wire.rs` - The previous architecture had an equivalent decision in ADR-024 (bidirectional call protocol with EventEnvelope framing), which is archived in the reference implementation at `/workspace/@alkdev/alknet-main/`. \ No newline at end of file diff --git a/docs/architecture/decisions/007-bistream-type-definition.md b/docs/architecture/decisions/007-bistream-type-definition.md index 53457b1..437a63f 100644 --- a/docs/architecture/decisions/007-bistream-type-definition.md +++ b/docs/architecture/decisions/007-bistream-type-definition.md @@ -116,4 +116,54 @@ The BiStream trait is a thin convenience — `AsyncRead + AsyncWrite + Send + Un - ADR-006: ALPN string convention and connection model - OQ-01: BiStream type definition (resolved by this ADR) - iroh ProtocolHandler pattern: `docs/research/references/iroh/iroh/` -- Pivot proposal: `docs/research/pivot/alpn-service-architecture.md` \ No newline at end of file +- Pivot proposal: `docs/research/pivot/alpn-service-architecture.md` + +## Amendments + +### Amendment 1 (2026-07-09): `Connection::from_stream` opens the server-side door + +This ADR's "WASM constraint" section argued that if `Connection` (or +BiStream) were bound to a QUIC library, WASM targets and alternative +transports couldn't implement it — and that a trait-based `BiStream` +preserves the *client-side* door. That argument was correct for `BiStream` +(the trait) but incomplete for `Connection`: until ADR-065, `Connection` +was a concrete type with only QUIC variants (`ConnectionKind::Quinn` / +`ConnectionKind::Iroh`), plus a `ConnectionKind::Mock` test stub. There was +no way to construct a `Connection` from a non-QUIC stream, which meant +TCP+TLS, SSH channels, WebTransport streams, and wasm streams could not +be dispatched through the `HandlerRegistry` — the *server-side* dispatch +door was closed. + +**Resolved by [ADR-065](065-connection-from-stream-generic-single-stream.md):** +`Connection::from_stream(send, recv, alpn, remote_addr)` and +`Connection::from_bidi(stream, alpn, remote_addr)` construct a `Connection` +from any `AsyncRead + AsyncWrite` pair. A new `ConnectionKind::Stream` +variant holds a single read/write pair behind a `Mutex>` with a +yield-once `accept_bi` contract (QUIC yields many streams; everything else +yields one, then `ConnectionClosed`). Every existing `ProtocolHandler` +works over the new kind **unchanged** — handlers that loop `accept_bi` +(TtyAdapter) get one iteration; handlers that call once (HttpAdapter) get +the stream directly. Both correct, no branching on transport. + +The stream-level `SendStreamKind::Mock` / `RecvStreamKind::Mock` variants +(already generic `Box` — the name was wrong) are +renamed to `Stream` and made load-bearing: `from_stream` calls +`SendStream::from_stream` / `RecvStream::from_stream`. + +### Amendment 2 (2026-07-09): `MockConnection` / `ConnectionKind::Mock` removed + +This ADR's Decision section and the `Connection` sketch referenced "test +mock" as one of the things `Connection` wraps. The implementation had a +`MockConnection` trait and `ConnectionKind::Mock` variant for test-only +full-connection mocks. ADR-065 removed both entirely: test stubs now use +`Connection::from_stream(tokio::io::sink(), tokio::io::empty(), alpn, +addr)` — `tokio::io::empty()` yields immediate EOF on read (handler exits +cleanly), and `accept_bi` returns `ConnectionClosed` after the first take +(run loop exits). One connection kind for production and tests, not two. +The "test mock" concept this ADR references is now subsumed by +`from_stream` — a test connection is just a single-stream connection with +EOF-on-read. + +The server-side WASM door (OQ-09) is no longer closed by `Connection` being +QUIC-bound — `from_stream` accepts any `AsyncRead + AsyncWrite`, including +wasm-compatible streams. See OQ-09 for the updated resolution. \ No newline at end of file diff --git a/docs/architecture/decisions/008-secret-service-integration.md b/docs/architecture/decisions/008-secret-service-integration.md index ef5ec68..f9a33a2 100644 --- a/docs/architecture/decisions/008-secret-service-integration.md +++ b/docs/architecture/decisions/008-secret-service-integration.md @@ -64,7 +64,7 @@ This is analogous to the reverse-proxy admin key pattern (ADR-028 in the reverse ## References - ADR-003: Crate decomposition (alknet-vault is standalone) -- ADR-005: irpc as call protocol foundation (for alknet-call; the vault no longer uses irpc — see ADR-025) +- ADR-005: irpc as call protocol foundation (superseded by ADR-064 — irpc was never integrated; the vault no longer uses irpc — see ADR-025) - ADR-009: One-way door decision framework - ADR-014: Secret material flow and capability injection (specifies the mechanism this ADR described in prose) - ADR-025: Vault local-only dispatch (dropped irpc from the vault; direct method calls only) diff --git a/docs/architecture/decisions/010-alpn-router-and-endpoint.md b/docs/architecture/decisions/010-alpn-router-and-endpoint.md index 5c55ca0..4b7450d 100644 --- a/docs/architecture/decisions/010-alpn-router-and-endpoint.md +++ b/docs/architecture/decisions/010-alpn-router-and-endpoint.md @@ -198,4 +198,60 @@ pub enum HandlerError { - iroh Router pattern: `docs/research/references/iroh/` - Reference implementation: `alknet-main/crates/alknet-core/src/server/serve.rs` - Reference stealth mode: `alknet-main/crates/alknet-core/src/server/stealth.rs` -- Reference iroh transport: `alknet-main/crates/alknet-core/src/transport/iroh_transport.rs` \ No newline at end of file +- Reference iroh transport: `alknet-main/crates/alknet-core/src/transport/iroh_transport.rs` + +## Amendments + +### Amendment 1 (2026-07-09): TCP+TLS can dispatch through the ALPN router via `from_stream` + +This ADR's Decision section states: **"TCP mode is not an endpoint concern."** +The rationale was that bare TCP (SSH over port 22) does not use QUIC or +ALPN, so TCP access is handled by individual handlers listening on a TCP +socket independently — a handler-specific concern, not a core endpoint +concern. + +That rationale holds for *bare TCP* (no TLS, no ALPN). But +**[ADR-065](065-connection-from-stream-generic-single-stream.md)** adds +`Connection::from_stream` / `from_bidi`, which construct a `Connection` +from any `AsyncRead + AsyncWrite` pair — including a +`TlsStream`. A TCP+TLS accept loop can now call +`Connection::from_bidi(tls_stream, alpn, remote_addr)` and dispatch through +the **same `HandlerRegistry`** as QUIC connections, by the ALPN negotiated +in the TLS handshake. This is not a parallel listener bypassing the core — +it's the same ALPN dispatch, over a non-QUIC transport. + +**Revised reading of "TCP is not an endpoint concern":** the +`AlknetEndpoint` struct (quinn + iroh) remains QUIC-only — the endpoint +does not own a TCP+TLS accept loop. But a TCP+TLS accept loop can be +constructed *outside* the endpoint (by the assembly layer or a handler) +and feed connections into the same `HandlerRegistry` the endpoint uses. +The endpoint is one accept-loop source; a TCP+TLS loop is another source +that shares the registry. The "not an endpoint concern" framing is +preserved at the struct level (no `tcp: Option` on +`AlknetEndpoint`); the "TCP can't participate in ALPN dispatch" framing +is **reversed** — `from_stream` is the primitive that lets TCP+TLS +participate without changing the endpoint design. + +The unblocked follow-ups (not part of ADR-065, but enabled by it): + +- **Standard HTTP over TCP+TLS** (`api.alk.dev`'s requirement): a TLS + accept loop wraps each `TlsStream` as a `Connection` via + `from_bidi` and dispatches to `HttpAdapter` by the negotiated ALPN + (`h2`/`http/1.1`). `HttpAdapter::handle` calls `accept_bi` once (yielded + by the single stream), then runs hyper over it — unchanged from the + QUIC path. No handler code changes. +- **SSH channel dispatch**: an SSH handler wraps each russh channel as a + `Connection` via `from_stream` and dispatches by channel-type (treated as + the ALPN string) through `HandlerRegistry`. One SSH connection carries + heterogeneous channels — a multiplexing power QUIC's per-connection ALPN + doesn't provide natively. +- **WebTransport stream dispatch** (parked per ADR-044, unblocked + structurally): the WT handler wraps each WT stream via `from_stream`. + +The `iroh 0.35 → 1.0.2` migration (commit `acd049e`, 2026-07-09) is a +related cleanup: it bumps the iroh dep to 1.0, unblocking `alknet-blobs` +(which pulls `iroh 1.0` transitively). It is not an architectural change — +6 API surface edits in `endpoint.rs` / `types.rs` (the `Endpoint::builder` +preset, `SecretKey::from_bytes`/`generate` signatures, +`Connection::remote_id`/`alpn` return types). No ADR needed; the endpoint +design is unchanged. \ No newline at end of file diff --git a/docs/architecture/decisions/018-vault-standalone-crate.md b/docs/architecture/decisions/018-vault-standalone-crate.md index 152809f..7ff4dbd 100644 --- a/docs/architecture/decisions/018-vault-standalone-crate.md +++ b/docs/architecture/decisions/018-vault-standalone-crate.md @@ -201,8 +201,9 @@ makes the freeze explicit and enforceable by review. ## References - ADR-003: Crate decomposition (alknet-vault is standalone) -- ADR-005: irpc as call protocol foundation (irpc remains the foundation - for alknet-*call*; the vault no longer uses irpc — see ADR-025) +- ADR-005: irpc as call protocol foundation (superseded by ADR-064 — irpc + was never integrated into alknet-call; the vault no longer uses irpc + either — see ADR-025) - ADR-025: Vault local-only dispatch (dropped irpc from the vault; the vault uses direct method calls, no actor, no remote capability) - ADR-008: Vault integration point (CLI-embedded, assembly-layer only) diff --git a/docs/architecture/decisions/025-vault-local-only-dispatch.md b/docs/architecture/decisions/025-vault-local-only-dispatch.md index 9e63b7d..4a89e7e 100644 --- a/docs/architecture/decisions/025-vault-local-only-dispatch.md +++ b/docs/architecture/decisions/025-vault-local-only-dispatch.md @@ -312,8 +312,11 @@ version of ADR-018's intent. ## References - ADR-005: irpc as call protocol foundation (this ADR amends the vault - reference in ADR-005's Decision and Consequences; irpc remains the - foundation for alknet-*call*, just not for alknet-*vault*) + reference in ADR-005's Decision and Consequences; ~~irpc remains the + foundation for alknet-*call*, just not for alknet-*vault*~~ — **this + claim is itself superseded by [ADR-064](064-irpc-never-integrated-hand-rolled-framing.md)**, + which records that irpc was never integrated into alknet-call either; + neither the vault nor the call protocol uses irpc) - ADR-008: Vault integration point (the vault is a capability source accessed at assembly time — this ADR makes that the *only* mode) - ADR-014: Secret material flow and capability injection (`DerivedKey` diff --git a/docs/architecture/decisions/064-irpc-never-integrated-hand-rolled-framing.md b/docs/architecture/decisions/064-irpc-never-integrated-hand-rolled-framing.md new file mode 100644 index 0000000..54976ef --- /dev/null +++ b/docs/architecture/decisions/064-irpc-never-integrated-hand-rolled-framing.md @@ -0,0 +1,164 @@ +# ADR-064: irpc Was Never Integrated — Hand-Rolled EventEnvelope Framing + +## Status + +Accepted + +## Context + +ADR-005 accepted "irpc as the call protocol foundation" based on the +previous architecture's use of irpc. When the call protocol was implemented, +it turned out that **no `.rs` file in the workspace ever imported irpc**. +The workspace `Cargo.toml` declared `irpc = "0.16"` / `irpc-derive = "0.16"` +as workspace dependencies, and `crates/alknet-call/Cargo.toml` declared +`irpc = { workspace = true }`, but the import was never written. The wire +protocol (`crates/alknet-call/src/protocol/wire.rs`) is hand-rolled +length-prefixed JSON — 4-byte big-endian length prefix + UTF-8 JSON body — +not an irpc service. + +This was discovered when building an external app against the crates: the +`irpc 0.16` workspace dep was a version-gap blocker for `alknet-blobs` (which +pulls `irpc 0.17` transitively via `iroh-blobs 0.103`). A grep for any `irpc` +import in the workspace found zero hits — the dep was dead weight carried +over from the previous architecture without verification. + +The framing, operation registry, dispatch, and subscription patterns that +ADR-005 attributed to irpc are all hand-rolled in alknet-call: + +- **Framing**: `FrameFramedReader` / `FrameFramedWriter` in `wire.rs` — + length-prefixed JSON, hand-written against `tokio::io::AsyncRead`/ + `AsyncWrite`. Not an irpc service. +- **Operation registry**: `OperationSpec`, `Handler`, `OperationRegistry`, + `AccessControl` — hand-rolled in alknet-call, not irpc's `Service` trait. +- **Event types**: `call.requested`, `call.responded`, `call.completed`, + `call.aborted`, `call.error` — the alknet call protocol's own event + vocabulary, not irpc's. +- **Subscription/streaming**: `StreamingHandler` / `invoke_streaming()` + (ADR-049) — hand-rolled, not irpc's streaming patterns. + +The `EventEnvelope { type, id, payload }` shape was derived from the +`@alkdev/pubsub` TypeScript `EventEnvelope` (`/workspace/@alkdev/pubsub/src/ +types.ts`), not from irpc. ADR-005's claim that "the wire format is irpc's +EventEnvelope framing" was wrong — irpc was never imported, and the envelope +shape has a different origin (the pubsub prior art, ADR-013). The framing +coincidentally resembles irpc's (both are length-prefixed JSON), which is +how the misattribution went unnoticed. + +### What ADR-005 got right + +Despite the irpc misattribution, ADR-005's *architectural* decisions are +correct and stand unchanged: + +- The call protocol uses length-prefixed JSON `EventEnvelope` framing + (hand-rolled, not irpc-supplied). +- The wire format is cross-language and consumable from TypeScript, Python, + any language (JSON is inherently cross-language — ADR-005's "mitigated: + it's length-prefixed JSON" note was the load-bearing point, not the irpc + attribution). +- Operations use JSON Schema discovery. The `OperationSpec` shape is + hand-rolled, JSON-Schema-compatible — the same property ADR-005 attributed + to irpc, achieved without irpc. + +### Why a new ADR rather than an amendment + +ADR-005's Decision and Consequences are built on the premise "alknet-call +uses irpc as its foundation — irpc IS the call protocol's core." That +premise is false. Amending ADR-005 to say "actually it's hand-rolled" would +leave an ADR whose Context, Decision, and Consequences sections all argue +for a choice that was never made. The correct record is: ADR-005 is +superseded; the call protocol uses hand-rolled framing (this ADR-064); the +architectural properties ADR-005 sought (proven framing, cross-language +JSON, streaming) are preserved, but the mechanism is hand-rolled, not +irpc-sourced. + +### The dead dep removal + +The `irpc` / `irpc-derive` workspace deps and the `alknet-call` consumer dep +were removed in commit `668d777` (2026-07-09). `irpc` may be re-added as +`0.17` when `alknet-blobs` lands (it pulls `irpc 0.17` transitively via +`iroh-blobs 0.103`), but that would be a *transitive* dependency of +`alknet-blobs`, not a direct dependency of `alknet-call` — alknet-call does +not import irpc and has no plans to. See +[`docs/research/transport-generalization/findings.md`](../../research/transport-generalization/findings.md) +§3.1 for the removal trace. + +## Decision + +1. **ADR-005 is superseded.** The call protocol does not use irpc. irpc was + never imported by any `.rs` file in the workspace. The dead `irpc` / + `irpc-derive` workspace and crate deps are removed. + +2. **The call protocol uses hand-rolled `EventEnvelope` framing.** The wire + format is length-prefixed JSON (4-byte big-endian length + UTF-8 JSON + body), implemented in `crates/alknet-call/src/protocol/wire.rs`. The + `EventEnvelope { type, id, payload }` shape was derived from the + `@alkdev/pubsub` TypeScript prior art (ADR-013), not from irpc. The + framing, operation registry, dispatch, and streaming patterns are all + hand-rolled in alknet-call. + +3. **The architectural properties ADR-005 sought are preserved by the + hand-rolled implementation:** + - Proven framing — length-prefixed JSON is a well-understood, + battle-tested pattern; the hand-rolled implementation is tested (207 + lib + 2 integration tests passing). + - Cross-language — JSON is inherently consumable from any language; + NAPI, WASM, and browser clients speak the same wire format. + - Streaming — `StreamingHandler` / `invoke_streaming()` (ADR-049) provide + the subscription/streaming patterns ADR-005 attributed to irpc, + hand-rolled. + +4. **irpc is not a planned dependency for alknet-call.** If `alknet-blobs` + pulls irpc transitively, it will be a transitive dependency of that + crate, not a direct dependency of alknet-call. alknet-call's framing, + registry, and dispatch are hand-rolled and will remain so. The "mitigated: + irpc is lightweight and we can fork if needed" caveat in ADR-005 is moot + — there is nothing to fork because nothing was integrated. + +5. **The vault's irpc drop (ADR-025) stands.** ADR-025 dropped irpc from + alknet-vault. With this ADR, irpc is also confirmed absent from + alknet-call. The vault and call decisions are now consistent: neither + crate uses irpc. The only difference is that ADR-025 *removed* a real + (but unused-for-its-primary-path) irpc dependency from the vault, while + this ADR records that alknet-call's irpc dependency was never integrated + at all — it was a Cargo.toml entry with no corresponding import. + +## Consequences + +**Positive:** +- The spec matches the code. ADR-005's irpc claims were a spec/code + divergence that surfaced only when the `irpc 0.16` version gap blocked + `alknet-blobs`. This ADR closes the divergence. +- `alknet-blobs` is unblocked — the dead `irpc 0.16` workspace dep is + gone; `iroh-blobs 0.103` (which pulls `irpc 0.17` transitively) no longer + conflicts with a workspace-pinned older irpc. +- The call protocol's framing, registry, and dispatch are documented + accurately as hand-rolled — readers of the spec aren't sent looking for + an irpc integration that doesn't exist. +- The cross-language story is unchanged: JSON wire format, JSON Schema + discovery. The mechanism changed (hand-rolled vs irpc), but the property + ADR-005 sought is preserved. + +**Negative:** +- The call protocol does not inherit irpc's testing or production pedigree + for its framing. Mitigation: length-prefixed JSON is a trivial, + well-understood pattern; the hand-rolled implementation is tested; and + the framing is small enough to audit completely (~30 lines in `wire.rs`). +- ADR-005's claim that "the call protocol inherits irpc's streaming and + subscription patterns" was wrong — those patterns are hand-rolled + (ADR-049). The streaming implementation is younger and less battle-tested + than irpc's would have been, but it is also simpler and fully owned. + +## References + +- ADR-005: irpc as call protocol foundation (superseded by this ADR) +- ADR-025: Vault local-only dispatch (dropped irpc from the vault; this ADR + records irpc was never integrated into alknet-call either) +- ADR-013: Rust as canonical implementation (the `@alkdev/pubsub` prior art + that the `EventEnvelope` shape was actually derived from) +- ADR-049: Streaming handler for subscriptions (the hand-rolled streaming + dispatch path) +- Call protocol wire format: `crates/alknet-call/src/protocol/wire.rs` +- Transport generalization findings: + [`docs/research/transport-generalization/findings.md`](../../research/transport-generalization/findings.md) + §3.1 (dead `irpc` dep removal) +- Removal commit: `668d777` (2026-07-09) \ No newline at end of file diff --git a/docs/architecture/decisions/065-connection-from-stream-generic-single-stream.md b/docs/architecture/decisions/065-connection-from-stream-generic-single-stream.md new file mode 100644 index 0000000..5aad8a9 --- /dev/null +++ b/docs/architecture/decisions/065-connection-from-stream-generic-single-stream.md @@ -0,0 +1,236 @@ +# ADR-065: Connection::from_stream — Generic Single-Stream Connections + +## Status + +Accepted + +## Context + +ADR-007 defines `Connection` as a concrete type wrapping a QUIC connection +(quinn or iroh). ADR-010 establishes the endpoint as a multi-connectivity +QUIC acceptor — quinn and iroh, both producing QUIC connections dispatched by +ALPN. The `ProtocolHandler` trait (ADR-002) receives a `Connection`, and +handlers call `accept_bi()` / `open_bi()` to get bidirectional streams. + +This design is **welded to QUIC**. Both real `ConnectionKind` variants +(`Quinn`, `Iroh`) are QUIC. The `HttpAdapter::handle` method calls +`connection.accept_bi().await` to get a bidi stream and serves HTTP over it +— "HTTP over QUIC," not "HTTP over TCP+TLS." There is no way to serve the +standard HTTP interface that `api.alk.dev` (an external app being built +against the crates) requires without either bypassing the +`HandlerRegistry` (a parallel listener, defeating the ALPN-router design) +or generalizing `Connection` to accept a non-QUIC stream. + +The same welding blocks `alknet-ssh` (needs to dispatch SSH channels — +each channel is a read/write pair — through the same `HandlerRegistry` as +QUIC connections) and WebTransport stream dispatch (each WT stream is a +read/write pair). The `TtyAdapter` and `CallAdapter` dispatch loops are +already transport-agnostic in their inner logic — only the +`connection.accept_bi()` call is QUIC-coupled, because `accept_bi` only +works when `Connection` is QUIC-kind. + +### The yield-once contract composes + +QUIC's `accept_bi` returns a new bidi stream per call (many). A generic +single-stream connection's `accept_bi` returns the underlying stream on the +first call, then `ConnectionClosed` on all subsequent calls. This is the +contract that makes the abstraction compose: + +- Handlers that loop `accept_bi` (TtyAdapter) get one session per + single-stream connection — the loop body runs once, then + `ConnectionClosed` breaks the loop. Correct. +- Handlers that call `accept_bi` once (HttpAdapter) get the stream + directly. Correct. + +No branching on transport. The handler code is unchanged across QUIC +(many streams) and TCP+TLS / SSH channels / WebTransport streams (one +stream). The `ProtocolHandler` trait shape is not touched — this is an +additive change to `Connection`, not a trait revision. + +### The stream-level Mock variants were already generic + +`SendStreamKind::Mock(Box)` and `RecvStreamKind::Mock(Box)` were already generic stream holders — the name was wrong +(carried over from a test-only context). The generalization renames them +to `Stream` and makes them load-bearing: `Connection::from_stream` calls +`SendStream::from_stream` / `RecvStream::from_stream` to wrap the halves of +the single stream. + +### The connection-level Mock is removed + +The findings doc (`docs/research/transport-generalization/findings.md`) +proposed keeping `ConnectionKind::Mock` / `MockConnection` for test-only +full-connection mocks. The implementation went further: `MockConnection` +and `ConnectionKind::Mock` are removed entirely. Test stubs that +previously used `Connection::from_mock(Arc)` now use +`Connection::from_stream(tokio::io::sink(), tokio::io::empty(), alpn, +addr)` — `tokio::io::empty()` yields immediate EOF on the read side, +causing the handler's `handle_stream` to exit cleanly, and `accept_bi` +returns `ConnectionClosed` after the first take (driving the run loop to +exit). This is simpler (one connection kind, not two) and the test stubs +are shorter. `from_stream` subsumes the test-mock use case because a test +connection is just a single-stream connection with EOF-on-read. + +### Why not change the ProtocolHandler trait + +An earlier analysis proposed changing `ProtocolHandler::handle` to take a +single `Channel` instead of a `Connection`, moving the multiplexing loop +from the handler to the endpoint. This ADR does **not** do that: + +1. **TtyAdapter already establishes the pattern.** The handler loops + `accept_bi` and dispatches each stream internally. SSH does the same — + parse channels, dispatch each. The multiplexing loop belongs in the + handler, not the endpoint. +2. **The trait shape is a one-way door (ADR-009).** Changing + `handle(Connection)` → `handle(Channel)` would require migrating every + handler and would lock in a specific multiplexing model. `from_stream` + is additive — it extends `Connection` without touching the trait. If a + trait change is ever warranted, it can come later; `from_stream` doesn't + preclude it. + +See `docs/research/transport-generalization/findings.md` §6 for the full +argument against the trait shape change. + +## Decision + +### Add `ConnectionKind::Stream` + +A new variant holding a single read/write pair behind a +`Mutex>` — the yield-once semantic. No +feature gate (generic, no transport deps). `StreamConn` is always +available; the quinn/iroh variants remain feature-gated. + +### Add `Connection::from_stream` and `Connection::from_bidi` + +```rust +/// Construct a Connection from a pre-split read/write pair. +/// `accept_bi()` yields this pair once, then returns `ConnectionClosed`. +/// `open_bi()` returns `StreamClosed` (a single stream can't open new streams). +pub fn from_stream( + send: impl AsyncWrite + Send + Unpin + 'static, + recv: impl AsyncRead + Send + Unpin + 'static, + alpn: Vec, + remote_addr: Option, +) -> Self; + +/// Convenience for a single bidirectional stream (e.g. TlsStream). +/// Splits internally via tokio::io::split. +pub fn from_bidi( + stream: impl AsyncRead + AsyncWrite + Send + Unpin + 'static, + alpn: Vec, + remote_addr: Option, +) -> Self; +``` + +### Make `accept_bi`'s yield-once contract explicit + +The `accept_bi` doc comment now states the transport semantics: QUIC yields +many streams, single-stream yields once then `ConnectionClosed`. This is +the contract that makes the abstraction compose — handlers don't branch +on transport. + +### Rename stream-level `Mock` → `Stream` + +`SendStreamKind::Mock` → `SendStreamKind::Stream`, +`RecvStreamKind::Mock` → `RecvStreamKind::Stream`. +`SendStream::from_mock` → `from_stream`, `RecvStream::from_mock` → +`from_stream`. The variants were already generic stream holders; the name +was wrong. Drop the `#[allow(dead_code)]` — `from_stream` is now +load-bearing. + +### Remove `MockConnection` / `ConnectionKind::Mock` + +The connection-level test mock trait and variant are removed. Test stubs +use `Connection::from_stream` with `tokio::io::sink()` / `tokio::io::empty()` +(immediate EOF on read → handler exits cleanly → `accept_bi` returns +`ConnectionClosed` → run loop exits). One connection kind for both +production and tests, not two. + +### `open_bi` on `Stream` returns `StreamClosed` + +A single stream cannot open new application streams. `open_bi` on +`ConnectionKind::Stream` returns `StreamError::StreamClosed`. Handlers that +call `open_bi` (the call protocol's server→client direction) work over +QUIC but not over a single-stream connection — this is inherent to the +transport, not a flaw. A handler that needs `open_bi` should not be +dispatched over a single-stream connection (or should multiplex its own +sub-streams within the one stream, as the call protocol does over a single +WebTransport stream). + +### What does NOT change + +- `ProtocolHandler` trait shape — `handle(&self, connection: Connection, + auth: &AuthContext)` stays. This is an additive change to `Connection`, + not a trait revision (ADR-009: the trait is a one-way door). +- `HandlerRegistry` — unchanged. +- All handler code (`HttpAdapter`, `TtyAdapter`, `CallAdapter`) — + unchanged. `HttpAdapter` is one `accept_bi` call away from + transport-agnostic (it already is — the call works over `from_stream`). +- `BiStream` trait — unchanged (ADR-007). `from_stream` is a server-side + connection constructor; `BiStream` is a client-side/test convenience + trait. They're complementary, not competing. +- The endpoint's accept loops (quinn/iroh) — unchanged. The TCP+TLS accept + loop that *uses* `from_stream` is a follow-up, not this ADR. + +## Consequences + +**Positive:** +- Every existing `ProtocolHandler` works over TCP+TLS, SSH channels, + WebTransport streams, and wasm streams **unchanged** — dispatch through + the same `HandlerRegistry` by ALPN string, no handler code changes. +- `api.alk.dev`'s HTTP blocker is resolved: a TCP+TLS accept loop can call + `Connection::from_bidi(tls_stream, alpn, remote_addr)` and dispatch + through `HandlerRegistry` — `HttpAdapter` works unchanged over the + single stream. (The accept loop itself is a follow-up commit; the + primitive it needs is now in place.) +- `alknet-ssh` is unblocked: the SSH handler wraps each russh channel via + `from_stream` and dispatches by channel-type (treated as the ALPN string) + through `HandlerRegistry`. One SSH connection carries heterogeneous + channels (`alknet/tty`, `alknet/call`, `h2`, ...) — a multiplexing power + QUIC's per-connection ALPN doesn't give natively. +- WebTransport stream dispatch is unblocked: the WT handler wraps each WT + stream via `from_stream` and dispatches through `HandlerRegistry` (the + primitive exists; WT itself is parked per ADR-044). +- The server-side WASM door (OQ-09) is no longer closed by `Connection` + being QUIC-bound — `from_stream` accepts any `AsyncRead + AsyncWrite`, + including wasm-compatible streams. (The accept-loop runtime remains + tokio-bound; the *connection* door is now open.) +- One connection kind for production and tests (no `MockConnection` + trait) — simpler type, shorter test stubs. +- No new deps, no `Cargo.toml` change — `tokio::io::split` is already + available via the existing tokio dep. + +**Negative:** +- `open_bi` on a single-stream connection returns `StreamClosed` — + handlers that need server→client stream initiation (the call protocol's + bidirectional call direction) don't work over a single-stream + connection. This is inherent to the transport, not a design flaw: a + single TCP+TLS stream is not a multiplexed transport. Handlers that need + `open_bi` should run over QUIC, or multiplex their own sub-streams within + the one stream (as the call protocol does over a single WebTransport + stream — the `EventEnvelope` framing is stream-agnostic, ADR-012). +- The `close()` method's `code`/`reason` args are QUIC-specific + (application-level close codes). For a raw stream they're ignored — the + drop is the close. This is the same best-effort semantic `close` already + had for the removed `Mock` variant. +- A `Mutex` on the `StreamConn` — a single lock per `accept_bi` / `close` + call. Negligible cost (one `take()`), but it is a lock where the QUIC + variants have none. + +## References + +- ADR-002: ProtocolHandler trait (unchanged by this ADR) +- ADR-007: BiStream type definition (amended by this ADR — `Connection` is + no longer QUIC-only; the server-side WASM door is open) +- ADR-009: One-way door decision framework (why the trait shape is not + changed — `from_stream` is additive) +- ADR-010: ALPN router and endpoint (amended by this ADR — "TCP is not an + endpoint concern" is revised; `from_stream` lets TCP+TLS participate in + ALPN dispatch via a handler-internal accept loop) +- ADR-012: Call protocol stream model (the `EventEnvelope` framing is + stream-agnostic — composes over `from_stream`) +- OQ-09: WASM target boundaries (resolution amended — the server-side + dispatch door is no longer closed by `Connection` being QUIC-bound) +- Transport generalization findings: + [`docs/research/transport-generalization/findings.md`](../../research/transport-generalization/findings.md) +- Implementation commit: `865fef6` (2026-07-09) \ No newline at end of file diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md index 665a349..78051a4 100644 --- a/docs/architecture/open-questions.md +++ b/docs/architecture/open-questions.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-07-08 +last_updated: 2026-07-09 --- # Open Questions @@ -171,6 +171,7 @@ filtering the tables above. - **Blocked on**: A concrete server-side WASM use case, or a deliberate confirmation that WASM stays a client-side design constraint. Tracked as `architecture/oq-09-wasm-server-use-case` in `tasks/architecture/`. - **Priority**: low +- **Amendment (2026-07-09)**: The `Connection` door is now open via `Connection::from_stream` (ADR-065) — a `Connection` can be constructed from any wasm-compatible stream. What remains closed is the **accept-loop runtime** (`tokio::spawn` does not run on WASM; `PendingRequestMap`/`CallAdapter` use tokio channels). The blocking condition (a concrete server-side WASM use case) is unchanged. - **Full file**: [OQ-09](questions/009-wasm-target-boundaries.md) ### OQ-10: Git Adapter Scope — Smart Protocol Only or Full Server? diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 834f333..b390d09 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-06-23 +last_updated: 2026-07-09 --- # Alknet Overview @@ -35,7 +35,7 @@ alknet-core │ └── StaticConfig, DynamicConfig (ArcSwap) │ ├── alknet-ssh (depends on alknet-core, russh) -├── alknet-call (depends on alknet-core, irpc) +├── alknet-call (depends on alknet-core) │ ├── CallAdapter (server: ProtocolHandler for alknet/call) │ ├── Call client (send/receive over QUIC) │ ├── OperationSpec, OperationRegistry, AccessControl @@ -95,7 +95,7 @@ See [ADR-002](decisions/002-protocol-handler-trait.md) and [ADR-007](decisions/0 | ALPN | Handler | Description | |------|---------|-------------| | `alknet/ssh` | SshAdapter | SSH-2 handshake, channel multiplexing, SOCKS5, port forwarding | -| `alknet/call` | CallAdapter | JSON-RPC via irpc: operations, streaming, pub/sub | +| `alknet/call` | CallAdapter | JSON-RPC via hand-rolled EventEnvelope framing: operations, streaming, pub/sub | | `alknet/git` | GitAdapter | Git smart protocol over QUIC (gix, pkt-line) | | `alknet/sftp` | SftpAdapter | SFTP protocol (russh-sftp core) | | `alknet/msg` | MessageAdapter | E2E encrypted messaging, mixnet | @@ -142,11 +142,11 @@ See [ADR-008](decisions/008-secret-service-integration.md) and [ADR-014](decisio ## Call Protocol -alknet-call uses irpc as its foundation. The wire format is length-prefixed JSON (EventEnvelope framing). Operations are registered in an irpc registry with JSON Schema discovery. The call protocol supports request/response, streaming subscriptions, and pub/sub. +alknet-call uses hand-rolled `EventEnvelope` framing (length-prefixed JSON). The wire format, operation registry, and dispatch are all hand-rolled in alknet-call — irpc was never integrated (ADR-064 supersedes ADR-005, which had accepted "irpc as the call protocol foundation" based on the previous architecture but was never implemented as stated). Operations are registered in a hand-rolled registry with JSON Schema discovery. The call protocol supports request/response, streaming subscriptions, and pub/sub. The call protocol's adapter contract (from_openapi, from_mcp, from_call, to_openapi, to_mcp) enables bidirectional composition — operations can be imported from external sources and exported to external protocols. These adapter traits are defined in Rust in alknet-call. The existing TypeScript `@alkdev/operations` library informed the design and may be adapted for browser use (see ADR-013). -See [ADR-005](decisions/005-irpc-as-call-protocol-foundation.md) for the full rationale. +See [ADR-064](decisions/064-irpc-never-integrated-hand-rolled-framing.md) for the full rationale (supersedes [ADR-005](decisions/005-irpc-as-call-protocol-foundation.md)). ## WASM Compatibility @@ -159,7 +159,7 @@ The following types live in alknet-core and are used across handler crates: | Type | Purpose | |------|---------| | `ProtocolHandler` | The trait every handler implements | -| `Connection` | QUIC connection (or mock) — handlers open/accept streams on it | +| `Connection` | Transport connection (QUIC via quinn/iroh, or a generic single stream via `from_stream` — ADR-065) — handlers open/accept streams on it | | `BiStream` | Trait: `AsyncRead + AsyncWrite + Send + Unpin` — bidirectional byte stream | | `AuthContext` | Resolved identity for a connection (may be partial) | | `Identity` | Authenticated peer identity (inbound) | @@ -195,7 +195,7 @@ All design decisions are documented as ADRs in [decisions/](decisions/). | [002](decisions/002-protocol-handler-trait.md) | ProtocolHandler Trait | One trait replaces StreamInterface/MessageInterface | | [003](decisions/003-crate-decomposition.md) | Crate Decomposition | One crate per protocol handler, core provides shared infra | | [004](decisions/004-auth-as-shared-core.md) | Auth as Shared Core | IdentityProvider in core, handlers extract credentials | -| [005](decisions/005-irpc-as-call-protocol-foundation.md) | irpc as Call Protocol Foundation | Call protocol uses irpc for registry, framing, dispatch | +| [005](decisions/005-irpc-as-call-protocol-foundation.md) | irpc as Call Protocol Foundation | ~~Accepted~~ → **Superseded** by [ADR-064](decisions/064-irpc-never-integrated-hand-rolled-framing.md) (irpc was never integrated) | | [006](decisions/006-alpn-convention-and-connection-model.md) | ALPN String Convention and Connection Model | `alknet/` prefix, one ALPN per connection | | [007](decisions/007-bistream-type-definition.md) | BiStream Type Definition | BiStream is a trait, handlers receive Connection not BiStream | | [008](decisions/008-secret-service-integration.md) | Vault Integration Point | CLI-embedded, vault is a capability source accessed at assembly time | diff --git a/docs/architecture/questions/009-wasm-target-boundaries.md b/docs/architecture/questions/009-wasm-target-boundaries.md index a7e299d..fa8a4b6 100644 --- a/docs/architecture/questions/009-wasm-target-boundaries.md +++ b/docs/architecture/questions/009-wasm-target-boundaries.md @@ -5,5 +5,33 @@ - **Door type**: One-way (when applicable) - **Priority**: low - **Blocked on**: A concrete server-side WASM use case, or a deliberate confirmation that WASM stays a client-side design constraint. Tracked as `architecture/oq-09-wasm-server-use-case` in `tasks/architecture/`. -- **Resolution**: Not an active question — WASM compatibility is a design constraint (see ADR-009, overview.md design principles), not a deliverable. Specific WASM targeting decisions will be made when individual crates are implemented. **BiStream being a trait preserves the *client-side* stream door** — a browser can implement BiStream over WebTransport streams. **The *server-side* dispatch door is NOT preserved by ADR-007 and is a known, accepted closure**: `Connection` is a concrete quinn-bound struct (not a trait), the accept loop uses `tokio::spawn` (tokio does not run on WASM), and the call-protocol dispatch internals (`PendingRequestMap`, `CallAdapter`) use tokio `oneshot`/`mpsc` channels. A WASM server-side peer would require a `Connection` trait and a runtime-abstracted accept loop — not planned. The browser path is client-side via a JS SDK, not server-side Rust-to-WASM. This is an explicit one-way door, not an oversight. -- **Cross-references**: ADR-007, ADR-009 +- **Resolution**: Not an active question — WASM compatibility is a design constraint (see ADR-009, overview.md design principles), not a deliverable. Specific WASM targeting decisions will be made when individual crates are implemented. **BiStream being a trait preserves the *client-side* stream door** — a browser can implement BiStream over WebTransport streams. **The *server-side* connection door is now open via `Connection::from_stream` (ADR-065):** `from_stream` accepts any `AsyncRead + AsyncWrite` pair, including wasm-compatible streams, so a `Connection` can be constructed from a wasm stream and dispatched through the `HandlerRegistry` like any QUIC connection. What *remains* closed is the **accept-loop runtime**: the `AlknetEndpoint` accept loops use `tokio::spawn` (tokio does not run on WASM), and the call-protocol dispatch internals (`PendingRequestMap`, `CallAdapter`) use tokio `oneshot`/`mpsc` channels. A WASM server-side peer would require a runtime-abstracted accept loop (not `tokio::spawn`) and a runtime-abstracted channel set — the `Connection` door is open, the runtime door is not. The browser path is client-side via a JS SDK, not server-side Rust-to-WASM. This is an explicit one-way door (the runtime), not an oversight; the `Connection` door was a one-way door that ADR-065 opened (additively, without a trait change). +- **Cross-references**: ADR-007 (Amendment 1 — `from_stream` opens the server-side door), ADR-009, ADR-065 + +### Amendment (2026-07-09) + +The original resolution (above) stated: "**The *server-side* dispatch door +is NOT preserved by ADR-007 and is a known, accepted closure**: +`Connection` is a concrete quinn-bound struct (not a trait), the accept +loop uses `tokio::spawn` (tokio does not run on WASM)..." That was accurate +when written: until ADR-065, `Connection` had only QUIC variants +(`ConnectionKind::Quinn` / `ConnectionKind::Iroh`) plus a +`ConnectionKind::Mock` test stub — no way to construct a `Connection` from a +non-QUIC stream. + +**ADR-065 opens the `Connection` door.** `Connection::from_stream` / +`from_bidi` accept any `AsyncRead + AsyncWrite` pair, including +wasm-compatible streams. A `Connection` can now be constructed from a wasm +stream and dispatched through the `HandlerRegistry` like any QUIC +connection — the *connection* door is open. The resolution text above has +been updated to reflect this. + +**What is still closed:** the **accept-loop runtime.** The +`AlknetEndpoint` accept loops use `tokio::spawn`, and the call-protocol +dispatch internals (`PendingRequestMap`, `CallAdapter`) use tokio +`oneshot`/`mpsc` channels. Tokio does not run on WASM. A WASM server-side +peer would require a runtime-abstracted accept loop and a +runtime-abstracted channel set — the `Connection` door is open, the +runtime door is not. This is the remaining one-way door, and it is still a +*runtime* door, not a *connection* door. The blocking condition (a concrete +server-side WASM use case) is unchanged. \ No newline at end of file