diff --git a/docs/architecture/crates/hub/README.md b/docs/architecture/crates/hub/README.md index 995e46a..7682d41 100644 --- a/docs/architecture/crates/hub/README.md +++ b/docs/architecture/crates/hub/README.md @@ -6,9 +6,8 @@ last_updated: 2026-07-09 # alknet-hub The hub pattern as a reusable crate: peer lifecycle management, connection -supervision, operation aggregation, and channel management across connected -peers. One QUIC connection per peer; channel 0 carries the call protocol as -the universal control plane; channels 1..N carry any ALPN as data planes. +supervision, operation aggregation, and service discovery across connected +peers. One QUIC connection per peer carrying the call protocol. ## What @@ -18,52 +17,17 @@ not introduce new types — it wires the existing `PeerCompositeEnv`, `CallClient`, `CallAdapter`, `from_call`, and `Dispatcher` types into a coherent hub runtime. -### The channel model - -A hub-worker connection is a single QUIC connection. QUIC multiplexes -multiple bidirectional streams over one connection. The hub uses this -multiplexing to carry multiple protocols over one connection: - -- **Channel 0 — call protocol (control plane).** Always `alknet/call`. This is - the bootstrap channel. It handles operation discovery (`from_call`), - operation routing (`invoke_peer`), service discovery (`services/list`, - `services/list-peers`), and channel negotiation (`channel/open`, - `channel/close`). Every hub-worker connection has channel 0. - -- **Channels 1..N — any ALPN (data planes).** Opened via `channel/open` on - the call protocol. Each channel is a bidirectional QUIC stream, wrapped as - a `Connection` via `Connection::from_bidi` (ADR-065), and handed to the - same `ProtocolHandler::handle(connection, auth)` that handles dedicated - connections. The handler does not know it's on a multiplexed channel — it - sees a `Connection` and dispatches normally. - -Channel negotiation is symmetric — either side can open a channel. A hub can -open a tty channel to a worker (hub wants a terminal on the worker), or a -worker can open a tty channel to the hub (worker exposes a terminal to the -hub). Same pattern as `from_call`: bidirectional, symmetric, negotiated over -the call protocol. - -The hub's role for channels 1..N is **channel management** — tracking which -channels are open on which peer, answering `channel/list`, and rejecting -duplicate `channel/open` requests. The hub does not interpret the protocol -running on the channel and does not proxy streams. What the caller does with -the resulting channel is the assembly layer's business. - ### What the hub provides 1. **Peer lifecycle** — dial, accept, disconnect, reconnect with backoff. Identity resolution via `IdentityProvider`. One connection per peer. 2. **Aggregated operation env** — a shared `PeerCompositeEnv` across all - calls (ADR-067). Operations discovered via `from_call` on channel 0 are - registered in each peer's connection overlay and aggregated into the - shared env. `invoke_peer` routes operation calls to the right peer. + calls (ADR-067). Operations discovered via `from_call` are registered in + each peer's connection overlay and aggregated into the shared env. + `invoke_peer` routes operation calls to the right peer. -3. **Channel management** — tracking which channels are open on which peer. - `channel/open` and `channel/close` operations on the call protocol. - Channel tracking for channels 1..N. - -4. **Service discovery** — `services/list-peers` returns each connected +3. **Service discovery** — `services/list-peers` returns each connected worker's operation list via `PeerCompositeEnv::peer_operations` (ADR-068). ## Why @@ -92,7 +56,7 @@ independently, `alknet-hub` provides it once, as a reusable crate. ### Hub struct The `Hub` is the central type. It owns the aggregated `PeerCompositeEnv`, the -`OperationRegistry`, the `Dispatcher`, and channel tracking: +`OperationRegistry`, and the `Dispatcher`: ```rust pub struct Hub { @@ -100,13 +64,6 @@ pub struct Hub { aggregated_env: Arc>, dispatcher: Dispatcher, identity_provider: Arc, - /// Open channels per peer: PeerId → set of channel numbers. - /// Channel 0 (call protocol) is tracked implicitly via the - /// CallConnection; channels 1..N are tracked here. The hub tracks - /// which channels exist so it can answer channel/list and reject - /// duplicate channel/open requests. The hub does not hold the - /// channel's Connection — the ProtocolHandler owns it. - channels: Arc>>>, } ``` @@ -129,7 +86,6 @@ impl Hub { aggregated_env, dispatcher, identity_provider, - channels: Arc::new(RwLock::new(HashMap::new())), } } @@ -145,12 +101,11 @@ impl Hub { The `Hub` exposes builder methods for optional hooks (`with_session_source`, `with_ownership_provider`, `with_timeout`) that delegate to the `Dispatcher`. -### Peer lifecycle (channel 0 bootstrap) +### Peer lifecycle -Every hub-worker connection starts with channel 0 — the call protocol. The -hub establishes channel 0, runs `from_call` to discover the peer's operations, -and attaches the peer to the aggregated env. After channel 0 is established, -either side can open additional channels via `channel/open`. +Every hub-worker connection carries the call protocol. The hub establishes the +connection, runs `from_call` to discover the peer's operations, and attaches +the peer to the aggregated env. #### Inbound workers (workers dial the hub) @@ -163,8 +118,8 @@ registers the bundles, and attaches the peer to the aggregated env: /// Callback invoked by CallAdapter::handle when a worker connects inbound. /// Fires after identity resolution and before the dispatch loop starts. /// Carries both on_connected (runs from_call, registers discovered ops, -/// attaches peer to aggregated env) and on_disconnected (detaches peer, -/// drops channel tracking on run_loop exit). +/// attaches peer to aggregated env) and on_disconnected (detaches peer +/// on run_loop exit). pub struct WorkerConnectedCallback { hub: Arc, config: FromCallConfig, @@ -204,10 +159,6 @@ The callback is wired into `CallAdapter` via a new builder method: ```rust impl CallAdapter { - /// Register a callback invoked when a worker connects inbound. - /// Fires after identity resolution and before the dispatch loop. - /// The callback runs from_call, registers discovered operations, - /// and attaches the peer to the aggregated env. pub fn with_worker_connected_callback( mut self, callback: WorkerConnectedCallback, @@ -226,7 +177,7 @@ The `CallAdapter::handle` flow becomes: bundles, attaches peer to aggregated env. 4. Run the dispatch loop (`run_loop`). 5. On `run_loop` exit, invoke `on_disconnected` on the callback — detaches - peer from aggregated env, drops channel tracking. + peer from aggregated env. The assembly layer constructs the callback and passes it to `CallAdapter`: @@ -237,11 +188,10 @@ let adapter = CallAdapter::new(registry, identity_provider) .with_worker_connected_callback(callback); ``` -> **Note on OQ-54**: The callback approach replaces the earlier "explicit -> post-hoc call" design. The callback fires inside `handle()` because -> `handle()` blocks until disconnect — there is no "after handle accepts" -> point for the assembly layer to hook into. The callback is the committed -> design; OQ-54 is resolved. +> **Note on OQ-54**: The callback fires inside `handle()` because `handle()` +> blocks until disconnect — there is no "after handle accepts" point for the +> assembly layer to hook into. The callback is the committed design; OQ-54 is +> resolved. #### Outbound workers (hub dials workers) @@ -252,9 +202,9 @@ configurable backoff: ```rust impl Hub { - /// Dial a worker, establish channel 0 (call protocol), discover its + /// Dial a worker, establish the call-protocol connection, discover its /// operations, and attach it to the aggregated env. Returns the - /// worker's PeerId and the live CallConnection (channel 0). + /// worker's PeerId and the live CallConnection. pub async fn dial_worker( &self, addr: SocketAddr, @@ -282,17 +232,13 @@ impl Hub { Ok((peer_id, connection)) } - /// Detach a peer from the aggregated env and drop all channel tracking. - /// Called on disconnect (both inbound and outbound). + /// Detach a peer from the aggregated env. Called on disconnect + /// (both inbound and outbound). fn on_worker_disconnected(&self, peer_id: &PeerId) { self.aggregated_env .write() .expect("aggregated env lock poisoned") .detach_peer(peer_id); - self.channels - .write() - .expect("channels lock poisoned") - .remove(peer_id); } /// Supervise an outbound worker: dial, discover, attach. On disconnect, @@ -345,73 +291,6 @@ impl Hub { } ``` -### Channel negotiation (channels 1..N) - -After channel 0 is established, either side can open additional channels via -the call protocol. The hub registers `channel/open`, `channel/close`, and -`channel/list` as built-in operations on its curated registry. - -#### channel/open - -Opens a new bidirectional stream on the QUIC connection, wraps it as a -`Connection` via `Connection::from_bidi`, and hands it to the -`HandlerRegistry` for the requested ALPN. The handler runs on a spawned task; -the channel is tracked until closed. - -``` -Request: { "alpn": "alknet/tty", "channel": 1 } -Response: { "channel": 1, "status": "open" } -``` - -The `channel` field is a hint — the caller suggests a channel number. The -receiver may accept it or assign a different number. Channel 0 is reserved -for the call protocol. - -The operation is symmetric: either side can call it. When the hub calls -`channel/open` on a worker, the worker opens the stream and dispatches to its -handler. When a worker calls `channel/open` on the hub, the hub opens the -stream and dispatches to its handler. - -#### channel/close - -Closes a previously opened channel. Drops the `Connection`, which cancels the -handler task and releases resources. - -``` -Request: { "channel": 1 } -Response: { "channel": 1, "status": "closed" } -``` - -#### channel/list - -Lists open channels on this connection. Channel 0 (call protocol) is always -present and always listed. - -``` -Response: { "channels": [{ "channel": 0, "alpn": "alknet/call" }, { "channel": 1, "alpn": "alknet/tty" }] } -``` - -#### Channel usage - -`channel/open` is a call-protocol operation. The hub provides it. What the -caller does with the resulting channel is the caller's business — the hub -crate does not prescribe a specific proxying pattern. - -The general mechanism: when `channel/open` is invoked, the receiver opens a -new QUIC stream on the connection, wraps it as `Connection::from_bidi(stream, -alpn, ...)`, and hands it to the `HandlerRegistry` for the requested ALPN. -The handler runs on the receiver side. The caller gets back a channel number -and can interact with the handler through the call protocol (e.g. by invoking -operations that the handler registered, or by opening a reciprocal channel). - -A common pattern (but not the only one): a hub client wants a tty session on -a worker. The hub calls `channel/open { alpn: "alknet/tty" }` on the worker. -The worker opens the stream, wraps it, and hands it to `TtyAdapter::handle()`. -The hub now has a tty session running on the worker. The hub can proxy the -raw stream to the client, or expose tty operations through the call protocol, -or both. The hub crate provides the `channel/open` mechanism; the assembly -layer decides the proxying pattern. - ### Backoff configuration ```rust @@ -460,12 +339,6 @@ pub enum HubError { Discovery(#[from] AdapterError), #[error("call client error: {0}")] Client(#[from] ClientError), - #[error("channel {channel} already open on peer {peer}")] - ChannelAlreadyOpen { peer: PeerId, channel: u32 }, - #[error("channel {channel} not found on peer {peer}")] - ChannelNotFound { peer: PeerId, channel: u32 }, - #[error("channel {channel} open failed: {reason}")] - ChannelOpenFailed { channel: u32, reason: String }, } ``` @@ -486,10 +359,6 @@ pub enum HubError { - **Worker provisioning.** The hub does not spawn workers, configure them, or manage their lifecycle beyond connection supervision. Worker provisioning is an assembly-layer concern. -- **Protocol interpretation for channels 1..N.** The hub manages channels - (tracking, listing, open/close). It does not interpret the protocol - running on the channel and does not proxy streams. What the caller does - with the resulting channel is the assembly layer's business. ## Crate dependencies @@ -515,9 +384,6 @@ A downstream hub (alkapi) uses `alknet-hub` like this: let registry = OperationRegistryBuilder::new() .with_local(agent_chat_spec(), agent_chat_handler, ...) .with_local(services_list_spec(), services_list_handler, ...) - .with_local(channel_open_spec(), channel_open_handler, ...) - .with_local(channel_close_spec(), channel_close_handler, ...) - .with_local(channel_list_spec(), channel_list_handler, ...) // ... other curated ops .build(); let registry = Arc::new(registry); @@ -559,7 +425,6 @@ so the assembly layer can wire it into `CallAdapter::with_aggregated_env`. | from_call is manual | [ADR-069](../../decisions/069-from-call-manual-free-function.md) | `from_call` is a free function; the hub calls it after connect | | Peer-graph routing model | [ADR-029](../../decisions/029-peer-graph-routing-model.md) | Peer-keyed overlays, `PeerRef` routing, `AccessControl`-based peer auth | | PeerEntry and Identity.id | [ADR-030](../../decisions/030-peerentry-and-identity-id-decoupling.md) | `PeerId` = `Identity.id` = `PeerEntry.peer_id` (stable) | -| Connection::from_stream / from_bidi | [ADR-065](../../decisions/065-connection-from-stream-generic-single-stream.md) | Channel streams wrap as `Connection`; any `ProtocolHandler` consumes them unchanged | ## Open Questions @@ -577,12 +442,6 @@ See [open-questions.md](../../open-questions.md) for full details. start. `handle()` blocks until disconnect, so there is no "after handle accepts" point for the assembly layer to hook into. The `WorkerConnectedCallback` is the committed design. -- **OQ-55** (open): `channel/open` operation — this is a new call-protocol - operation. It needs a spec in `alknet-call` (the operation shape, the - stream-opening mechanism, the handler dispatch). The hub registers it as a - built-in operation; the implementation lives in `alknet-call`. The channel - model is committed; the operation spec is deferred to the call-protocol - implementation phase. ## References @@ -593,7 +452,6 @@ See [open-questions.md](../../open-questions.md) for full details. - [operation-registry.md](../call/operation-registry.md) — `OperationRegistry`, `OperationRegistryBuilder`, `OperationEnv` - ADR-029: Peer-Graph Routing Model -- ADR-065: `Connection::from_stream` — Generic Single-Stream Connections - ADR-067: Aggregated Peer-Environment Wiring - ADR-068: PeerCompositeEnv::peer_operations Override - ADR-069: from_call Is a Manual Free Function diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md index 4c8744e..b7d66fa 100644 --- a/docs/architecture/open-questions.md +++ b/docs/architecture/open-questions.md @@ -165,7 +165,6 @@ Door type is separate from whether a decision is made. A two-way door is a decis | [OQ-52](questions/052-callconnection-wait-for-close.md) | CallConnection::wait_for_close() for supervision loop | open | two | med | | [OQ-53](questions/053-backoff-config-defaults.md) | BackoffConfig default policy | open | two | low | | [OQ-54](questions/054-inbound-worker-hook-placement.md) | Inbound worker on_worker_connected hook placement | resolved | two | low | -| [OQ-55](questions/055-channel-open-operation.md) | channel/open call-protocol operation | open | two | med | ## Deferred / Blocked diff --git a/docs/architecture/questions/055-channel-open-operation.md b/docs/architecture/questions/055-channel-open-operation.md deleted file mode 100644 index 5fc8bc5..0000000 --- a/docs/architecture/questions/055-channel-open-operation.md +++ /dev/null @@ -1,52 +0,0 @@ -# OQ-55: channel/open Call-Protocol Operation - -- **Origin**: [crates/hub/README.md](crates/hub/README.md) -- **Status**: open -- **Door type**: Two-way -- **Priority**: medium -- **Resolution**: Not yet decided. - -The hub's channel model requires a `channel/open` operation on the call -protocol. This operation opens a new bidirectional QUIC stream on the -existing connection, wraps it as a `Connection` via -`Connection::from_bidi` (ADR-065), and hands it to the `HandlerRegistry` -for the requested ALPN. The operation is symmetric — either side can call it. - -The operation shape is committed: - -``` -Request: { "alpn": "alknet/tty", "channel": 1 } -Response: { "channel": 1, "status": "open" } -``` - -What is not yet decided: - -- **Where the operation lives.** The `channel/open` handler needs access to - the underlying QUIC connection to open a new stream. This is a - `CallConnection` concern — the handler needs a reference to the connection - it's running on. Options: (a) the handler is registered with a closure - capturing the `CallConnection`, (b) the `OperationContext` carries a - reference to the connection, (c) a new `ConnectionHandle` type that the - handler can use to open streams. - -- **Channel number assignment.** The caller suggests a channel number; the - receiver may accept or reassign. The reassignment policy (first-available, - caller-always-wins, receiver-always-wins) is not decided. - -- **Handler dispatch on the receiving side.** When a `channel/open` request - arrives, the receiver opens the stream, wraps it as `Connection::from_bidi`, - and dispatches to the `HandlerRegistry` for the requested ALPN. The - dispatch mechanism (spawn a task, register in the endpoint's accept loop, - or a separate channel dispatch path) is not decided. - -- **`channel/close` and `channel/list`.** These are companion operations. - `channel/close` drops the `Connection` and cancels the handler task. - `channel/list` returns the set of open channels. Their shapes are - straightforward; the implementation depends on the `channel/open` - mechanism. - -The channel model is committed. The operation spec is deferred to the -call-protocol implementation phase — it depends on the `channel/open` -mechanism decision. - -- **Cross-references**: ADR-065, [crates/hub/README.md](crates/hub/README.md)