alkdev/alknet
6
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(arch): channels substrate simplification + stream_type decomposition + sub-crate split

Browse Source 
Three simplifications to the channels spec, all flowing from the review:

1. Substrate simplification (ADR-071 revised): the 9-byte chunk header is
   used in ALL substrates — in-line (TCP+TLS, WebTransport), native (QUIC
   bidi streams), and multi-connection. The ChannelsAdapter reads headers
   off every bidi stream it accepts; the transport's native multiplexing
   is a performance optimization (independent flow-control windows), not a
   protocol change. One wire format, one code path, one handler experience.
   The channel_id in the header is the correlation key across substrates.

2. Stream_type decomposition (ADR-071 revised): every stream_type is
   unidirectional. Bidirectionality is two stream_types (write + read), not
   one 'bidirectional' stream_type. Grouped in threes: 0/1/2 = data
   write/read/err, 3/4/5 = control write/read/err, % 3 formula. This
   resolves the TTY control channel's 'not actually bidirectional' flaw —
   control is now 3 (write, client→server) + 4 (read, server→client), each
   with its own reassembly buffer and EOF. Channel 0 uses [0,1] (call frames
   bidirectional via 0=in, 1=out). TTY uses [0,1,2,3,4]. ADR-072, 073, 074,
   077 updated for the new stream_type assignments.

3. Sub-crate decomposition (ADR-081 new): channels-core (pure multiplexer —
   wire format, demux/mux, ChannelBidiStreamSource, ChannelManager; depends
   on alknet-core only, no call dependency, ALPN-blind) / channels-call
   (channel 0 pre-negotiation + lifecycle op registrations; depends on
   channels-core + alknet-call) / channels-hub (relay) / channels-worker
   (ChannelClient). Isolates the call-protocol coupling from the pure
   multiplexer so the dependency graph is honest.

ADR-077 (TTY inside channels) updated: TTY now uses 5 sub-streams [0,1,2,3,4]
with control properly bidirectional via 3/4; amends ADR-052's stream_type
assignments for direct mode too (direct alknet/tty now uses 0-4, not 0-3).

Spec docs updated: channels-wire, channels-connection, channels-adapter,
channel-operations, overview, README.
This commit is contained in:
glm-5.2 committed 2026-07-12 15:47:42 +00:00
1 parent 2313c51f12
commit fd83fc1685
14 files changed
+734 -251
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/architecture/decisions/071-channels-wire-format.md
+187 -74
View File
@@ -2,14 +2,15 @@
## Status
Accepted
Accepted (revised 2026-07-12: substrate simplification + stream_type
decomposition)
## Context
`alknet-channels` is a multiplexing proxy: a `ProtocolHandler` on
`alknet/channels` that decomposes a single bidirectional transport stream into
N logical channels, each carrying a different ALPN. The wire format is the
substrate that makes one transport stream carry many channels.
`alknet/channels` that carries N logical channels, each with a different
ALPN, over transport stream(s). The wire format is the substrate that makes
this work.
Two prior formats inform this design:
@@ -18,16 +19,47 @@ Two prior formats inform this design:
encrypted transport stream.
2. **alknet-tty's chunk format (ADR-052)** — `[stream_type: u8][length: u32 be]
[payload]`, a fixed set of four sub-streams (stdin/stdout/stderr/control)
within one bidi stream. Validated by two POCs (alknet-docker-poc,
alknet-tty-poc) and in production code (`crates/alknet-tty/src/wire.rs`).
within one bidi stream. Validated by two POCs and in production code.
The channels format is the generalization: add a `channel_id: u32` prefix to
TTY's 5-byte header, turning a fixed 4-channel multiplexer into an arbitrary
N-channel multiplexer. The de-risk POC (`docs/research/alknet-channels/poc-
summary.md`, 28 tests) validated this is a clean generalization: the sync core
(`parse_header`/`write_header`) is pure and WASM-compatible by construction;
the mpsc-bridged async shell scales to N concurrent channels with per-channel
order preservation and cross-channel isolation.
N-channel multiplexer. The de-risk POC (28 tests) validated the core
mechanics.
### The substrate simplification
Different transports have different native multiplexing capabilities. QUIC
has native bidi streams; TCP+TLS does not. The initial framing of this ADR
treated the 9-byte header as "the in-line substrate format" — used when the
transport has no native multiplexing, with a separate substrate for QUIC
native streams. That framing adds complexity (two substrates, a
`channel_id`↔stream-ID mapping) for no benefit.
The simplification: **the 9-byte header is used in all substrates, on every
bidi stream that carries channels.** The `channel_id` in the header is the
logical correlation key. The transport's native multiplexing (when present)
is a performance optimization (independent flow-control windows per
stream), not a protocol change. The `ChannelsAdapter` reads the 9-byte
header off every bidi stream it accepts — on a transport with one stream
(in-line), the header demuxes N channels from that stream; on a transport
with N streams (QUIC native), each stream carries one logical channel and
the header provides `stream_type` and `channel_id` correlation. Same code
path, same wire format, same handler experience.
### The stream_type decomposition
The initial framing had `stream_type` 3 as "bidirectional" (control
messages). This is a design flaw: one stream_type both sides write to is not
properly multiplexed — it loses independent flow control, independent EOF,
and clean separation of concerns. The TTY crate's control channel already
exhibits this problem ("the control channel isn't actually bidirectional…
the adapter ignores Exit from the client" — phase-0 findings §Less
Straightforward Parts).
The fix: **every stream_type is unidirectional.** Bidirectionality is
achieved by having two stream_types (one write, one read), the same way
QUIC bidi streams are two unidirectional halves. Control becomes 3 (write,
client→server) and 4 (read, server→client), not one "bidirectional" 3.
## Decision
@@ -37,47 +69,119 @@ order preservation and cross-channel isolation.
[channel_id: u32 be][stream_type: u8][length: u32 be][payload bytes]
```
9 bytes of header. The `channel_id` is the addition over TTY's 5-byte format;
`stream_type` and `length` are identical to TTY's fields (ADR-052), preserving
the framing-disambiguation soundness property (§5 below).
9 bytes of header. The `channel_id` is the addition over TTY's 5-byte
format; `stream_type` and `length` are identical to TTY's fields (ADR-052),
preserving the framing-disambiguation soundness property.
| field | width | meaning |
|-------|-------|---------|
| `channel_id` | u32 BE | The logical channel this chunk belongs to. Channel 0 is pre-negotiated as `alknet/call` (ADR-072). Channels 1..N are opened dynamically via `channel/open`. |
| `stream_type` | u8 | The sub-stream within the channel. 0=stdin/data-in, 1=stdout/data-out, 2=stderr/diagnostic (optional), 3=control (JSON), 4-255 reserved. |
| `stream_type` | u8 | The unidirectional sub-stream within the channel. See "Stream types" below. |
| `length` | u32 BE | The payload length in bytes. 0 = EOF sentinel (same convention as TTY — ADR-052 §Sentinels). |
### `MAX_CHUNK_LEN`
`16 * 1024 * 1024` (16 MiB), matching TTY's cap (ADR-052 §5). A chunk with
`length > MAX_CHUNK_LEN` returns `ChunkTooLarge` and does not corrupt the
stream — the demux drops the chunk and continues. This preserves the framing-
disambiguation soundness property: the header is always exactly 9 bytes, so
the demux can always resync after a dropped chunk by reading the next 9-byte
stream — the demux drops the chunk and continues. The header is always
exactly 9 bytes, so the demux can always resync by reading the next 9-byte
header.
### Stream types (per channel)
### Stream types — unidirectional, grouped in threes
| stream_type | direction | purpose |
|-------------|-----------|---------|
| 0 | write half | data flowing in (stdin equivalent) |
| 1 | read half | data flowing out (stdout equivalent) |
| 2 | read half (optional) | error/diagnostic output (stderr equivalent) |
| 3 | bidirectional | control messages (ALPN-specific JSON) |
| 4-255 | reserved | future sub-stream types |
**Every stream_type is unidirectional.** Bidirectionality is two
stream_types (write + read), not one "bidirectional" stream_type. The
stream_types are grouped in threes:
Not all channels use all sub-streams. A TTY session uses 0-3. A raw tunnel
uses 0 and 1. An SSH connection uses 0 and 1 (SSH multiplexes internally).
The active `stream_type` set is declared at `channel/open` time
(ADR-073) and fixed for the channel's lifetime.
| Group | stream_type | direction | purpose |
|-------|-------------|-----------|---------|
| Data | 0 | write (client→server) | data in (stdin equivalent) |
| | 1 | read (server→client) | data out (stdout equivalent) |
| | 2 | read (server→client) | data err (stderr equivalent, optional) |
| Control | 3 | write (client→server) | control in (resize, signal, eof) |
| | 4 | read (server→client) | control out (exit, keepalive response) |
| | 5 | read (server→client) | control err (optional) |
| Future | 6/7/8 | write/read/read | next group, same pattern |
| | 9/10/11 | write/read/read | next group |
| | ... | | |
**Formula:** `stream_type % 3 == 0` → write half (in), `stream_type % 3 ==
1` → read half (out), `stream_type % 3 == 2` → diagnostic read half (err).
256 values / 3 = 85 groups. The `u32` channel_id space (~4 billion channels
before wrap, ADR-076) combined with 85 stream_type groups is effectively
unlimited for the intended use cases.
**Why unidirectional:** each stream_type gets its own reassembly buffer, its
own flow control, its own EOF. The TTY control channel becomes *actually*
bidirectional because there are two unidirectional streams (3 in, 4 out),
not one stream both sides write to. This resolves the "control channel
isn't actually bidirectional" problem the TTY crate has today. The same
principle applies to any future channel type — control is two halves, not
one shared stream.
### Per-ALPN stream_type sets
| ALPN | Active stream_types | Why |
|------|---------------------|-----|
| `alknet/call` (channel 0) | [0, 1] | call frames bidirectional via 0=in, 1=out |
| `alknet/tty` | [0, 1, 2, 3, 4] | data in/out/err + control in/out |
| `alknet/tunnel` | [0, 1] | data in/out only (no channels-layer control needed) |
| `alknet/ssh` | [0, 1] | SSH multiplexes internally, including its own control |
The active set is declared at `channel/open` time (ADR-073 `stream_types`
field) and fixed for the channel's lifetime. A tunnel that wants keepalive
could declare [0, 1, 3, 4].
### Substrate modes — same wire format, different stream counts
The 9-byte header is used in all substrates, on every bidi stream. The
difference between substrates is only **how many bidi streams the transport
yields**:
| Substrate | Transport | Streams | Header role |
|-----------|-----------|---------|--------------|
| In-line | TCP+TLS, WebTransport session, SSH `direct-tcpip` | 1 | Header demuxes N channels from that 1 stream |
| Native | QUIC (quinn/iroh) | N | Each stream carries 1 logical channel; header provides `stream_type` + `channel_id` correlation |
| Multi-connection | Any, N connections | N × M | Each connection is self-contained (own channel 0, own demux); header is per-connection |
The `ChannelsAdapter::handle` loop: `accept_bi()` → for each stream, read
the 9-byte header → route by `(channel_id, stream_type)` → reassemble. On
an in-line transport, `accept_bi()` yields once then `ConnectionClosed` —
the header does all the demux. On QUIC, `accept_bi()` yields repeatedly —
each stream is a channel, and the header provides `stream_type` and
`channel_id` correlation. Same code path, same wire format, same handler
experience.
**Why keep the header on QUIC when the stream already separates channels:**
- **`stream_type` decomposition.** TTY needs 5 sub-streams. On QUIC, one
stream per channel + the header carrying `stream_type` is simpler than 5
streams per channel and matches the in-line case's shape. Per-stream-type
flow control is handled at the reassembly buffer level (ADR-076)
regardless of substrate.
- **`channel_id` correlation.** The hub relay forwards a channel from the
browser leg to the spoke leg, mapping `browser_id ↔ spoke_id`. If
`channel_id` is in the header on every stream, the relay correlates by
reading the header — regardless of substrate. If `channel_id` lived only
in the `channel/open` response, the relay would need a stream-ID-to-
`channel_id` mapping per leg.
- **Uniform handler experience.** The handler receives a `Connection` and
calls `accept_bi()` or `into_sub_streams()`. It doesn't know or care
whether the substrate is in-line or native — the
`ChannelBidiStreamSource` wraps the reassembled stream either way.
The "bloat" (9 bytes/chunk on transports that have native multiplexing) is
the cost of uniformity. For the intended use cases (TTY, SSH, tunnels, call
operations), this is noise. The high-throughput escape hatch is
multi-connection, not stripping the header.
### Framing disambiguation (carried from ADR-052 §5)
Channel 0 is just another channel — its chunks have `channel_id=0` in the
header. Disambiguation between channel 0 (call protocol) and data channels is
by `channel_id`, not by a special first-byte trick. Within a channel,
`stream_type` 0 (stdin) from the server is invalid, so `0x00` as the first
byte of a chunk payload from the server is unambiguous.
header. Disambiguation between channel 0 (call protocol) and data channels
is by `channel_id`, not by a special first-byte trick. Within a channel,
`stream_type` 0 (write half) from the server is invalid, so `0x00` as the
first byte of a chunk payload from the server is unambiguous.
### Zero-length sentinel = EOF
@@ -87,61 +191,70 @@ is the clean-shutdown signal for a `(channel_id, stream_type)` pair.
### Sync core / async shell split
The wire format's core is pure byte manipulation — `parse_header(&[u8; 9]) ->
ChunkHeader` and `write_header(channel_id, stream_type, length, &mut [u8; 9])`.
No async, no platform dependencies. Compiles under `wasm32-unknown-unknown`
(validated by the POC). The async shell (demux/mux) wraps this core with
`read_exact`/`write_all` on the transport and `mpsc` routing. This split is
the same pattern as TTY's (REQ-TTY-01 generalization) and keeps the
WASM-compatible core separate from the tokio-dependent shell.
The wire format's core is pure byte manipulation — `parse_header(&[u8; 9])
-> ChunkHeader` and `write_header(channel_id, stream_type, length, &mut
[u8; 9])`. No async, no platform dependencies. Compiles under
`wasm32-unknown-unknown` (validated by the POC). The async shell (demux/mux)
wraps this core with `read_exact`/`write_all` on the transport and `mpsc`
routing. This split keeps the WASM-compatible core separate from the
tokio-dependent shell.
## Consequences
**Positive:**
- One multiplexing model replaces three (connection-level ALPN, stream-level
QUIC native, sub-stream-level TTY chunks). The hub's relay logic becomes
channel-by-channel byte forwarding, not per-protocol framing parsers.
- The 9-byte overhead is negligible for the intended use cases (TTY sessions,
SSH, tunnels, call operations). The format is a 4-byte extension to a
proven 5-byte format.
- One wire format across all substrates. The `ChannelsAdapter` code path is
the same regardless of transport; the transport's native multiplexing is a
performance optimization, not a protocol change.
- Every stream_type is unidirectional — proper multiplexing with independent
flow control and EOF per half. The TTY control channel becomes actually
bidirectional (3 in, 4 out), resolving the "not actually bidirectional"
flaw.
- The hub relay correlates by `channel_id` in the header, uniformly across
substrates — no stream-ID-to-`channel_id` mapping per leg.
- WASM-compatible by construction — the pure core has no platform deps.
- The framing-disambiguation property from ADR-052 carries forward unchanged.
**Negative:**
- All channels on one `alknet/channels` connection share one transport
stream's flow-control window. A slow consumer on one channel can
backpressure others. This is mitigated by bounded-buffer backpressure
(ADR-076) but not eliminated. For high-throughput bulk transfer, the
application uses N independent channels connections — same trade-off as
HTTP/2-over-TLS vs HTTP/3-over-QUIC. This is a transport property, not a
channels-format property.
- 9 bytes per chunk is 4 bytes more than TTY's 5-byte format. For
high-frequency small-chunk workloads (e.g., typing in a terminal), this is
a 80% header overhead increase. In practice the chunk size is driven by
the write pattern (a terminal sends a few bytes per keystroke regardless),
and the 4-byte delta is noise next to the TLS/QUIC overhead.
- 9 bytes per chunk on all substrates, including QUIC where the transport
already separates streams. This is 4 bytes more than using the QUIC stream
ID directly as the `channel_id`. For the intended use cases this is noise;
for high-throughput bulk transfer, the answer is multi-connection, not
stripping the header.
- All channels on one in-line connection share one transport stream's
flow-control window. A slow consumer on one channel can backpressure
others. Mitigated by bounded-buffer backpressure (ADR-076), not
eliminated. The native substrate (QUIC streams) avoids this — each
channel gets its own flow-control window. For high-throughput, use native
or multi-connection.
## Door type
**One-way.** The chunk header layout (`channel_id:u32 + stream_type:u8 +
length:u32`) is a wire-format commitment. Changing field widths, order, or
semantics after deployments exist requires a version migration. The
`stream_type` assignments (0=stdin, 1=stdout, etc.) are one-way for the same
reason — they are inherited from ADR-052 and preserved.
length:u32`) and the stream_type group assignments (0/1/2 = data, 3/4/5 =
control, `% 3` formula) are wire-format commitments. Changing them after
deployments exist requires a version migration.
The `MAX_CHUNK_LEN` value (16 MiB) is a two-way-door implementation detail
within the one-way format — it can be changed without a wire-format version
bump as long as both ends agree (it's a validation threshold, not a field
width).
within the one-way format.
## References
- ADR-052: alknet-tty wire format (the 5-byte format this generalizes)
- ADR-065: `Connection::from_stream` (the transport-agnostic Connection this
format rides on)
- ADR-052: alknet-tty wire format (the 5-byte format this generalizes;
amended by ADR-077 — scoped to direct TTY)
- ADR-065: `Connection::from_stream` (the transport-agnostic Connection)
- ADR-070: `BidiStreamSource` trait (the extension point the channels
connection implements)
- `docs/research/alknet-channels/poc-summary.md` — the POC that validated the
format (28 tests, WASM compile check)
- `docs/research/alknet-channels/phase-0-findings.md` §The Wire Format
- `crates/alknet-tty/src/wire.rs` — the 5-byte format implementation
connection implements; its docstring already anticipated per-channel
streams)
- ADR-072: channel 0 pre-negotiated (now uses stream_types [0, 1])
- ADR-073: channel lifecycle operations (stream_types field examples
updated)
- ADR-074: ChannelBidiStreamSource (into_sub_streams returns unidirectional
handles)
- ADR-076: backpressure (bounded-buffer applies at reassembly regardless of
substrate)
- ADR-077: TTY inside channels (5 sub-streams; control properly
bidirectional via 3/4)
- `docs/research/alknet-channels/poc-summary.md` — the POC that validated
the format (28 tests, WASM compile check)
- `docs/research/alknet-channels/phase-0-findings.md` §The Wire Format, §Less
Straightforward Parts (the control-channel bidirectionality problem)
docs/architecture/decisions/072-channel-0-pre-negotiated-call.md
+14 -7
View File
@@ -64,14 +64,21 @@ exactly as it does on a top-level `alknet/call` connection.
### Channel 0's stream_type usage
| stream_type | purpose |
|-------------|---------|
| 0 | `EventEnvelope` frames (JSON, length-prefixed — the call protocol wire format) |
| 1-255 | reserved for future call-protocol sub-streams |
| stream_type | direction | purpose |
|-------------|-----------|---------|
| 0 | write (client→server) | `EventEnvelope` frames from the client (call.requested, call.aborted) |
| 1 | read (server→client) | `EventEnvelope` frames from the server (call.responded, call.completed, call.error) |
Channel 0 uses only `stream_type` 0. The reservation of `stream_type`
1-255 is a future-proofing detail, not a current commitment — the call
protocol is JSON-only and single-stream by design (ADR-064).
Channel 0 uses stream_types [0, 1] — the call protocol is bidirectional via
two unidirectional halves, the same way every channel type works
(ADR-071 §stream_type decomposition). The call protocol's `(SendStream,
RecvStream)` pair maps directly: `SendStream` backed by stream_type 0,
`RecvStream` backed by stream_type 1. Both sides write to their write half
and read from their read half — no shared stream_type both sides write to.
The call protocol is JSON-only and single-stream by design (ADR-064).
stream_types 2-255 on channel 0 are reserved for future call-protocol
sub-streams.
## Consequences
docs/architecture/decisions/073-channel-lifecycle-operations.md
+16 -10
View File
@@ -50,7 +50,7 @@ Request (`call.requested` on channel 0):
"operation": "channel/open",
"input": {
"alpn": "alknet/tty",
"stream_types": [0, 1, 2, 3],
"stream_types": [0, 1, 2, 3, 4],
"params": { "backend": "docker", "cmd": ["bash"], "container": "abc123" },
"direction": "initiator-to-responder"
}
@@ -60,8 +60,8 @@ Request (`call.requested` on channel 0):
| field | type | meaning |
|-------|------|---------|
| `alpn` | string | The ALPN the channel will carry. The responder looks this up in its `HandlerRegistry`. |
| `stream_types` | `[u8]` | Which sub-stream types this channel will use. Declared at open time so both sides size reassembly. E.g. `[0,1,2,3]` for TTY, `[0,1]` for a tunnel. |
| `params` | object | ALPN-specific parameters. For `alknet/tty` this is the `NegotiateRequest`. For `alknet/tunnel` this is the target resource. The channels layer does not interpret `params` — it hands the JSON to the handler. |
| `stream_types` | `[u8]` | Which sub-stream types this channel will use. E.g. `[0,1,2,3,4]` for TTY (data in/out/err + control in/out), `[0,1]` for a tunnel, `[0,1]` for channel 0 (call frames). See ADR-071 §stream_type decomposition. |
| `params` | object | ALPN-specific parameters. For `alknet/tty` this is the `NegotiateRequest`. For `alknet/tunnel` this is the target resource. The channels layer does not interpret `params`. |
| `direction` | string | `initiator-to-responder` or `responder-to-initiator`. See "Direction semantics" below. |
Response (`call.responded`):
@@ -70,7 +70,7 @@ Response (`call.responded`):
{
"output": {
"channel_id": 7,
"stream_types": [0, 1, 2, 3]
"stream_types": [0, 1, 2, 3, 4]
}
}
```
@@ -230,14 +230,20 @@ alternative.
| Control path | When | Examples |
|--------------|------|----------|
| Call operations on channel 0 (`channel/control`, `channel/close`) | Control that doesn't need ordering relative to data, or lifecycle events | resize, signal, keepalive, close |
| `stream_type 3` chunks on the data channel | Control that MUST be ordered relative to data | EOF before exit, flush before close |
| `stream_type 3` (write, client→server) and `stream_type 4` (read, server→client) chunks on the data channel | Control that MUST be ordered relative to data | EOF before exit, flush before close |
The TTY crate's exit-chunk-is-last invariant (ADR-055) is the canonical
example of data-ordered control — it rides on `stream_type 3` because it
must arrive after the last stdin chunk, guaranteed by chunk ordering within
`(channel_id, stream_type)`, not by a call-protocol round-trip. The
`channel/close` operation that follows is on channel 0 and is ordered after
the data pump completes (REQ-CH-06).
example of data-ordered control — the exit message rides on `stream_type 4`
(read, server→client) because it must arrive after the last data on
`stream_type 1`, guaranteed by per-stream_type ordering. The client's EOF
signal rides on `stream_type 3` (write, client→server), ordered after the
last data on `stream_type 0`. The `channel/close` operation that follows is
on channel 0 and is ordered after the data pump completes (REQ-CH-06).
**Every stream_type is unidirectional** (ADR-071 §stream_type decomposition).
Control is bidirectional via two halves (3 in, 4 out), not one shared
stream_type both sides write to. This resolves the TTY control channel's
"not actually bidirectional" flaw.
## Consequences
docs/architecture/decisions/074-channelconnection-bidistreamsource.md
+31 -13
View File
@@ -74,21 +74,30 @@ session — exactly as the POC's `EchoHandler` and `TtyAdapter` do today.
### Sub-stream accessor for typed destructure (OQ-CH-10)
Some handlers need access to `stream_type` 2 (stderr) and 3 (control) in
addition to the main 0/1 pair. The `Connection` interface alone (accept_bi)
only exposes the 0/1 pair. The channels crate provides a typed-accessor
extension:
Some handlers need access to `stream_type` 2 (stderr), 3 (control in), and
4 (control out) in addition to the main 0/1 pair. The `Connection`
interface alone (accept_bi) only exposes the 0/1 pair. The channels crate
provides a typed-accessor extension:
```rust
// In alknet-channels:
pub struct ChannelSubStreams {
pub streams: Vec<(u8, SendStream, RecvStream)>,
/// (stream_type, handle) for each active stream_type. Each handle is
/// unidirectional: write stream_types (0, 3, 6, ...) carry a SendStream;
/// read stream_types (1, 2, 4, 5, 7, ...) carry a RecvStream.
/// See ADR-071 §stream_type decomposition.
pub streams: Vec<(u8, SubStreamHandle)>,
}
pub enum SubStreamHandle {
Send(SendStream), // write half (stream_type % 3 == 0)
Recv(RecvStream), // read half (stream_type % 3 == 1 or 2)
}
impl ChannelBidiStreamSource {
/// Returns the typed sub-streams for this channel, keyed by stream_type.
/// Consumes the source — call this instead of accept_bi() if the handler
/// needs direct access to stream_types 2/3. For handlers that only need
/// needs direct access to stream_types 2/3/4. For handlers that only need
/// the main 0/1 pair, accept_bi() is the path (and sub_streams() is not
/// called).
pub fn into_sub_streams(self) -> ChannelSubStreams { ... }
@@ -101,15 +110,24 @@ its typed names:
```rust
// In alknet-tty (inside-channels mode, ADR-077):
let sub = channel_source.into_sub_streams();
let mut stdin = sub.get(0); // SendStream
let mut stdout = sub.get(1); // RecvStream
let mut stderr = sub.get(2); // RecvStream (optional)
let mut control = sub.get(3); // RecvStream (JSON control)
let stdin = sub.get_send(0).unwrap(); // SendStream (write, client→server)
let stdout = sub.get_recv(1).unwrap(); // RecvStream (read, server→client)
let stderr = sub.get_recv(2); // Option<RecvStream> (read, optional)
let ctrl_in = sub.get_send(3).unwrap(); // SendStream (write, client→server)
let ctrl_out = sub.get_recv(4).unwrap();// RecvStream (read, server→client)
```
**The channels crate does not know about TTY's `stream_type` semantics.** It
exposes `(stream_type, SendStream, RecvStream)` tuples. The handler crate
maps stream_types to its typed names. This preserves ADR-003's
**Every stream_type is unidirectional** (ADR-071). Write stream_types
(`% 3 == 0`) carry a `SendStream`; read stream_types (`% 3 == 1 or 2`) carry
a `RecvStream`. There is no "bidirectional" stream_type — bidirectionality
is two halves (e.g., control is 3 write + 4 read). This resolves the TTY
control channel's "not actually bidirectional" flaw: the TTY adapter reads
exit/keepalive from `ctrl_out` (stream_type 4) and writes resize/signal/eof
to `ctrl_in` (stream_type 3), each with its own flow control and EOF.
**The channels crate does not know about TTY's `stream_type` semantics.**
It exposes `(stream_type, SubStreamHandle)` tuples. The handler crate maps
stream_types to its typed names. This preserves ADR-003's
no-handler-depends-on-another-handler rule and keeps the channels crate
ALPN-blind.
docs/architecture/decisions/075-channelsadapter-and-channelmanager.md
+24 -14
View File
@@ -39,27 +39,37 @@ impl ProtocolHandler for ChannelsAdapter {
async fn handle(&self, connection: Connection, auth: &AuthContext)
-> Result<(), HandlerError>
{
// 1. One bidi stream carries all channels.
// 1. Channel 0 is pre-negotiated as alknet/call (ADR-072).
// The first bidi stream the transport yields is channel 0.
let (send, recv) = connection.accept_bi().await?;
// 2. Channel 0 is pre-negotiated as alknet/call (ADR-072).
// Construct its reassembly buffers, wrap as a Connection via
// from_source(ChannelBidiStreamSource), and hand to the
// CallAdapter (looked up in the registry, same as every ALPN).
self.manager.preinstall_channel_0(send, recv, auth).await?;
// 3. Run the demux loop: read 9-byte headers, route payloads to
// per-(channel_id, stream_type) reassembly buffers.
self.manager.run_demux_loop(recv).await
// 2. Accept remaining bidi streams and read 9-byte headers off each.
// On an in-line transport (TCP+TLS, WebTransport), accept_bi()
// yields once and the header demuxes N channels from that stream.
// On QUIC native, accept_bi() yields repeatedly — each stream
// carries one logical channel, and the header provides
// stream_type + channel_id correlation. Same code path, same
// wire format (ADR-071 §substrate modes).
self.manager.run_demux_loop(connection).await
}
}
```
The `preinstall_channel_0` step is the only special case: it constructs the
reassembly buffers for `channel_id = 0`, wraps them as a `Connection` (via
`Connection::from_source` with a `ChannelBidiStreamSource` — ADR-074), and
hands that `Connection` to the `CallAdapter` — exactly as if `alknet/call`
had been the top-level ALPN. The `CallAdapter` is none the wiser.
The `preinstall_channel_0` step constructs the reassembly buffers for
`channel_id = 0` using stream_types [0, 1] (ADR-072), wraps them as a
`Connection` (via `Connection::from_source` with a `ChannelBidiStreamSource`
— ADR-074), and hands that `Connection` to the `CallAdapter` — exactly as if
`alknet/call` had been the top-level ALPN. The `CallAdapter` is looked up in
the same `HandlerRegistry` as every other ALPN.
`run_demux_loop` continues accepting bidi streams from the transport. For
each stream, it reads 9-byte headers and routes payloads to the matching
`(channel_id, stream_type)` reassembly buffer. On an in-line transport,
there is only one stream (channel 0 rides inside it via the header); the
header demuxes all channels. On QUIC, each subsequent stream is a new
channel; the header's `channel_id` correlates it. The loop is the same;
only the transport's stream count differs.
### `ChannelManager` — the shared state
docs/architecture/decisions/077-tty-inside-channels.md
+50 -18
View File
@@ -36,8 +36,8 @@ The `TtyAdapter` operates in two modes, determined by how it receives its
| Mode | When | Wire format | How the adapter gets sub-streams |
|------|------|-------------|---------------------------------|
| **Direct (`alknet/tty` ALPN)** | Top-level QUIC/TCP connection with ALPN `alknet/tty` | TTY's 5-byte format (ADR-052) | `accept_bi()` → parse 5-byte chunks → split into stream_types 0-3 |
| **Inside channels** | `channel/open` with ALPN `alknet/tty` on a channels connection | Channels' 9-byte format (ADR-071) — the channels layer de-chunks | `into_sub_streams()` (ADR-074) → four named `SendStream`/`RecvStream` pairs for stream_types 0-3 |
| **Direct (`alknet/tty` ALPN)** | Top-level QUIC/TCP connection with ALPN `alknet/tty` | TTY's 5-byte format (ADR-052, amended — see below) | `accept_bi()` → parse 5-byte chunks → split into stream_types 0-4 |
| **Inside channels** | `channel/open` with ALPN `alknet/tty` on a channels connection | Channels' 9-byte format (ADR-071) — the channels layer de-chunks | `into_sub_streams()` (ADR-074) → five named handles for stream_types 0-4 |
In both modes, the `TtyBackend` trait and `TtyHandle` are unchanged
(ADR-053). The backend allocates a PTY and returns a `TtyHandle`; the
@@ -45,16 +45,44 @@ adapter pumps data between the handle and the sub-streams. The difference is
only in how the adapter gets the sub-streams — 5-byte chunk parsing (direct)
vs. `into_sub_streams()` (channels).
### The control channel is now properly bidirectional
The phase-0 findings flagged that the TTY control channel "isn't actually
bidirectional… the adapter ignores Exit from the client." The root cause:
`stream_type 3` was "bidirectional" — one stream both sides wrote to, which
is not properly multiplexed.
ADR-071's stream_type decomposition fixes this: **every stream_type is
unidirectional.** Control is now two halves:
| stream_type | direction | purpose |
|-------------|-----------|---------|
| 3 | write (client→server) | control in: resize, signal, eof |
| 4 | read (server→client) | control out: exit, keepalive response |
The TTY adapter writes resize/signal/eof to `ctrl_in` (stream_type 3) and
reads exit/keepalive from `ctrl_out` (stream_type 4). Each has its own
reassembly buffer, its own flow control, its own EOF. The control channel
is *actually* bidirectional — two unidirectional streams, not one shared
stream both sides write to.
This amends ADR-052's stream_type assignments for direct mode too: direct
`alknet/tty` connections now use stream_types [0, 1, 2, 3, 4] (data in/out/
err + control in/out), not [0, 1, 2, 3]. The 5-byte format's `stream_type`
field gains value 4; the `ControlMessage` enum is unchanged (the JSON shape
is the same; the stream_type it rides on splits from 3 into 3+4).
### What changes in alknet-tty
1. **The adapter's session-driving code splits into two entry points:**
- `drive_session_direct(send, recv, backends, ...)` — the existing path:
parse 5-byte chunks, split into stream_types, pump. Used for direct
parse 5-byte chunks, split into stream_types 0-4, pump. Used for direct
`alknet/tty` connections.
- `drive_session_channels(sub_streams, backends, ...)` — the new path:
receive `ChannelSubStreams` (four named `SendStream`/`RecvStream`
pairs), pump directly without chunk parsing. Used when the channel's
`Connection` is backed by `ChannelBidiStreamSource`.
receive `ChannelSubStreams` (five named handles: stdin=SendStream,
stdout=RecvStream, stderr=Option<RecvStream>, ctrl_in=SendStream,
ctrl_out=RecvStream), pump directly without chunk parsing. Used when
the channel's `Connection` is backed by `ChannelBidiStreamSource`.
2. **The `TtyAdapter::handle()` branches on the `Connection`'s source type.**
The channels crate's `ChannelBidiStreamSource` is a `BidiStreamSource`
@@ -73,20 +101,24 @@ vs. `into_sub_streams()` (channels).
connections." The channels path does not use it. This amends ADR-052's
scope — the format is not replaced, it's scoped.
4. **The control channel (stream_type 3) works the same in both modes.** In
direct mode, control JSON rides in 5-byte chunks with `stream_type=3`. In
channels mode, control JSON rides in 9-byte chunks with `stream_type=3`
— but the channels layer de-chunks it, so the adapter reads raw JSON
bytes from its `control` `RecvStream` in both cases. The
`ControlMessage` enum (resize, signal, eof, exit) is unchanged.
4. **The control channel works the same in both modes, now properly
bidirectional.** In direct mode, control-in JSON rides in 5-byte chunks
with `stream_type=3` and control-out rides with `stream_type=4`. In
channels mode, control-in rides in 9-byte chunks with `stream_type=3`
(write) and control-out with `stream_type=4` (read) — but the channels
layer de-chunks them, so the adapter reads raw JSON bytes from
`ctrl_in`/`ctrl_out` in both cases. The `ControlMessage` enum (resize,
signal, eof, exit) is unchanged — the JSON shape is the same; only the
stream_type assignments change (3 splits into 3+4).
5. **The exit-chunk-is-last invariant (ADR-055) generalizes.** In direct
mode, the exit chunk is the last 5-byte chunk before stream close
(ADR-055). In channels mode, the exit control message is the last data on
`stream_type 3` before `channel/close` is sent on channel 0
(ADR-073 §channel/close). The ordering invariant is the same — exit
before close — but the mechanism differs: 5-byte chunk ordering (direct)
vs. `stream_type 3` ordering + `channel/close` after pump completion
mode, the exit chunk is the last 5-byte chunk on `stream_type=4` (read,
server→client) before stream close (ADR-055, amended). In channels mode,
the exit control message is the last data on `stream_type=4` before
`channel/close` is sent on channel 0 (ADR-073 §channel/close). The
ordering invariant is the same — exit before close — but the mechanism
differs: 5-byte chunk ordering on stream_type 4 (direct) vs.
`stream_type 4` ordering + `channel/close` after pump completion
(channels, REQ-CH-06).
### What does NOT change
docs/architecture/decisions/081-channels-subcrate-decomposition.md
+210
View File
@@ -0,0 +1,210 @@
# ADR-081: alknet-channels Sub-Crate Decomposition
## Status
Accepted
## Context
The initial channels spec (ADR-071-080) framed `alknet-channels` as a
single crate depending on both `alknet-core` and `alknet-call`. The
dependency on `alknet-call` arises because channel lifecycle operations
(`channel/open`, `channel/close`, `channel/control`,
`channel/resources/subscribe`) register on the call protocol's
`OperationRegistry`, and channel 0 is pre-negotiated as `alknet/call`
(ADR-072).
This creates two issues:
1. **The dependency graph conflates the pure multiplexer with the call-
protocol coupling.** The wire format, demux/mux, and
`ChannelBidiStreamSource` are ALPN-blind and call-protocol-blind — they
depend on `alknet-core` only. The channel 0 pre-negotiation and lifecycle
op registrations are the call-protocol coupling. Baking both into one
crate means any consumer that wants the multiplexer also pulls in the
call-protocol coupling, even if they don't need channel 0 to be
`alknet/call`.
2. **The "no special-casing for downstream crates" principle is
violated.** The user's constraint: "we don't want to be doing anything
special for downstream crates unless it's needed/useful to the point of
they all kind of benefit." The call-protocol coupling is a channels-crate
concern (channels needs call for orchestration), not a downstream-crate
concern. Separating them makes the dependency graph honest: the pure
multiplexer has no opinion about channel 0; the call-protocol coupling
is isolated where it belongs.
The substrate simplification (ADR-071 revised) strengthened this: the wire
format is the same across all substrates, and the `ChannelsAdapter` is
substrate-agnostic. The pure multiplexer (`channels-core`) is cleanly
separable from the call-protocol orchestration (`channels-call`).
## Decision
### Three crates
```
alknet-channels-core — the pure multiplexer (wire format, demux/mux,
│ ChannelBidiStreamSource, ChannelManager). Depends
│ on alknet-core only. ALPN-blind, call-protocol-blind,
│ transport-blind.
├── alknet-channels-call — channel 0 pre-negotiation + lifecycle op
│ │ registrations on the call protocol's
│ │ OperationRegistry. Depends on channels-core +
│ │ alknet-call.
│ ├── alknet-channels-hub — the relay (ADR-079). Depends on
│ │ channels-call. The hub-side role.
│ └── alknet-channels-worker — ChannelClient (ADR-080). Depends on
│ channels-call. The client/worker-side role.
```
### `alknet-channels-core`
The pure multiplexer. Contains:
- The 9-byte chunk wire format (`parse_header` / `write_header` — ADR-071)
- The demux/mux (`Demux`, `MuxHandle`/`MuxRunner` — ADR-075)
- `ChannelBidiStreamSource` (implements `BidiStreamSource` — ADR-074)
- `ChannelSubStreams` / `SubStreamHandle` (the typed destructure accessor)
- `ChannelManager` (the shared state — channel_id → ChannelState, but
**without** the `call_ops: Arc<OperationRegistry>` field; the manager is
ALPN-blind and call-protocol-blind)
- `ChannelsAdapter` (the `ProtocolHandler` on `alknet/channels` — the
read/demux loop, substrate-agnostic per ADR-071 revised)
Depends on `alknet-core` only. No `alknet-call` dependency. No opinion about
what channel 0 carries — that's the consumer's concern.
The `ChannelsAdapter::handle` in `channels-core` does NOT preinstall channel
0 as `alknet/call`. It runs the demux loop and routes chunks by
`channel_id`. Channel 0 is just another channel; what ALPN it carries is
determined by the consumer (the `channels-call` crate pre-negotiates it as
`alknet/call`; a hypothetical other consumer could pre-negotiate it
differently).
### `alknet-channels-call`
The call-protocol coupling. Contains:
- Channel 0 pre-negotiation as `alknet/call` (ADR-072) — the
`preinstall_channel_0` logic that constructs channel 0's reassembly
buffers with stream_types [0, 1] and hands the `Connection` to the
`CallAdapter`.
- The four lifecycle operations (ADR-073): `channel/open`,
`channel/close`, `channel/control`, `channel/resources/subscribe` —
registered on the call protocol's `OperationRegistry` at assembly time.
- `ChannelOperations` (the registration helper that closes over a
`ChannelManager` clone).
Depends on `channels-core` + `alknet-call`. This is where the
call-protocol coupling lives, isolated from the pure multiplexer.
### `alknet-channels-hub` and `alknet-channels-worker`
The two roles. These may be sub-crates or feature-gated modules within
`channels-call`; the exact packaging is a two-way-door implementation
detail. The contract is:
- **`channels-hub`** (or the hub role): the relay (ADR-079). Translates
`channel/open` on channel 0, byte-forwards data channels with
`channel_id` rewrite. Depends on `channels-call` (for the call-protocol
translation) + the hub's own `CallAdapter` / `from_call` machinery.
- **`channels-worker`** (or the worker/client role): `ChannelClient`
(ADR-080). Dials a transport, establishes the channels connection, runs
the demux/mux, exposes `open_channel(alpn, params) -> Channel`. Depends
on `channels-call` (for channel 0 pre-negotiation) + `alknet-call`'s
`CallClient`-equivalent.
The naming (hub/worker vs server/client) is a two-way-door detail. The
user noted "server/client" is muddy because a hub/worker can act as both
depending on the use case (bidirectionality — ADR-073 §direction
semantics). The roles are "the side that relays" (hub) and "the side that
dials" (worker/client), but both can open channels in either direction.
### What moves where
| Component | Original (ADR-071-080) | Now |
|-----------|------------------------|-----|
| 9-byte wire format | `alknet-channels` | `channels-core` |
| Demux/Mux | `alknet-channels` | `channels-core` |
| `ChannelBidiStreamSource` | `alknet-channels` | `channels-core` |
| `ChannelSubStreams` | `alknet-channels` | `channels-core` |
| `ChannelManager` (without `call_ops`) | `alknet-channels` | `channels-core` |
| `ChannelsAdapter` (demux loop only) | `alknet-channels` | `channels-core` |
| Channel 0 pre-negotiation | `alknet-channels` (ADR-072) | `channels-call` |
| `channel/open`/`close`/`control`/`resources/subscribe` ops | `alknet-channels` (ADR-073) | `channels-call` |
| `ChannelOperations` registration helper | `alknet-channels` | `channels-call` |
| Hub relay (ADR-079) | `alknet-channels` (spec) / `alknet-hub` (impl) | `channels-hub` (or `alknet-hub` — see below) |
| `ChannelClient` (ADR-080) | `alknet-channels` | `channels-worker` |
### Relationship to `alknet-hub`
The existing `alknet-hub` crate (`docs/architecture/crates/hub/README.md`)
is the hub pattern: peer lifecycle, aggregated env, service discovery. The
channels hub relay (ADR-079) is a channels-specific concern that the hub
crate consumes. The split: `channels-hub` (or the hub role within
`channels-call`) provides the relay logic; `alknet-hub` wires it into the
hub runtime alongside the call-protocol peer management. Whether the relay
lives in a `channels-hub` sub-crate or directly in `alknet-hub` is a
packaging decision — the contract (ADR-079) is the same either way. The
user's preference for decomposing channels into core/hub/worker suggests a
`channels-hub` sub-crate that `alknet-hub` depends on, but this is not
one-way and can be revisited during implementation.
## Consequences
**Positive:**
- The dependency graph is honest: `channels-core` is the pure multiplexer
with no call dependency; the call-protocol coupling is isolated in
`channels-call`. A consumer that wants the multiplexer without the
call-protocol orchestration can depend on `channels-core` only.
- The "no special-casing for downstream crates" principle is preserved:
`channels-core` doesn't know about `alknet-call`, `alknet-tty`, or any
handler crate. The call-protocol coupling is a channels-crate concern,
not a downstream-crate concern.
- The sub-crate split matches the existing pattern (`alknet-tty` +
`alknet-tty-local`, `alknet-docker` + its `tty` feature) — core in one
crate, consumer-specific wiring in another.
- The hub and worker roles are separated, matching the user's
decomposition preference and the bidirectionality of the channels
protocol (both sides can open channels; the roles are about who relays
vs who dials, not about request/response direction).
**Negative:**
- Three crates instead of one. The assembly layer must depend on
`channels-core` + `channels-call` (and optionally `channels-hub` or
`channels-worker`) instead of one `alknet-channels`. This is the cost of
the clean separation; the assembly layer already wires multiple crates,
so this is consistent with the existing pattern.
- The `ChannelsAdapter` in `channels-core` doesn't preinstall channel 0 —
the consumer does. This means `channels-core`'s `ChannelsAdapter::handle`
exposes a hook (callback or trait method) for the consumer to install
channel 0. `channels-call` provides the `preinstall_channel_0`
implementation; a different consumer could provide a different one. This
is a slightly more complex adapter shape than "channel 0 is always call,"
but it's the cost of the clean separation.
## Door type
**One-way (crate structure) + two-way (role packaging).** The three-crate
split (`channels-core` / `channels-call` / roles) is one-way — once
consumers depend on `channels-core` without `channels-call`, re-merging
them is a breaking change. The hub/worker packaging (separate sub-crates
vs feature-gated modules within `channels-call`) is two-way — an
implementation detail that can change without breaking the contract.
## References
- ADR-003: crate decomposition (no-handler-depends-on-another-handler —
preserved; the channels sub-crates depend on core/call, not on handlers)
- ADR-071: channels wire format (revised — substrate simplification; the
wire format is in `channels-core`)
- ADR-072: channel 0 pre-negotiated (moves to `channels-call`)
- ADR-073: channel lifecycle operations (move to `channels-call`)
- ADR-074: ChannelBidiStreamSource (in `channels-core`)
- ADR-075: ChannelsAdapter and ChannelManager (split: core demux in
`channels-core`, call coupling in `channels-call`)
- ADR-079: hub relay (in `channels-hub` or `alknet-hub`)
- ADR-080: ChannelClient (in `channels-worker`)
- `docs/architecture/crates/hub/README.md` — the existing hub crate (the
relay's consumer)