--- status: draft last_updated: 2026-07-18 --- # stream-unification — Findings: the stream_type convention and the TTY/channels convergence **Status:** Draft findings, iterating. Per the research-then-sync pattern, this doc iterates in `docs/research/`; we fix inter-document drift here, then sync to `docs/architecture/` and the ADRs only after it settles. **Scope:** The multiplexing layer — the `stream_type` space within a channel. This is *above* the transport leaf (ADR-092, settled) and *below* the channel-protocol layer (ADR-072/073). The transport leaf (`BiStream` as the handler-facing duplex type) is settled in ADR-092 and is not re-litigated here. **Date:** 2026-07-18 --- ## Layering (to keep the questions separate) | Layer | Question | Status | |-------|----------|--------| | Transport leaf | What type does `accept_bi` return? How does a handler get a duplex byte stream? | **Settled — ADR-092** (drafted, pushed). `accept_bi` returns `BiStream`; `from_bidi` is the only public stream constructor; the split never crosses a crate boundary as part of a constructor. | | Multiplexing (this doc) | How are `stream_type` values assigned within a channel? What's the addressing convention? | **In progress.** ADR-071's mod-3 group framing vs the proposed mod-2/mod-4 instance framing. | | Channel protocol | How are channels opened/closed? What's channel 0? | **Settled — ADR-072/073.** Channel 0 = `alknet/call` (hardcoded); channels 1..N opened via `channel/open`. Not re-litigated here. | The two findings-doc questions that got conflated in the previous draft, separated: - **Transport-leaf split/recombine** ("can `tokio::io::join`/`split` recombine halves from different sources?") — ADR-092's layer. Answer: yes, stdlib `tokio::io::join`/`split` is a pure type combinator; the halves don't need to come from the same source. Settled; not this doc's concern. - **Stream_type split/recombine** ("if stderr is its own instance with an unused write half, does the unused half cause problems?") — this doc's layer. Answer (verified in the existing POC code, `alknet-channels-poc/src/demux.rs:91-109, 161-181` and `mux.rs:58-63, 152-177`): the demux/mux is per-`stream_type` independent — there is **no pairing assumption** in the mechanism. An unused `stream_type` is an idle mpsc channel, not a wart. The mod-2 framing is trivially clean. **No POC needed** — the load-bearing property is already proven by the 28-test POC. --- ## The core question: stream_type assignment convention The mechanism (per-`stream_type` independent demux/mux, validated by the channels POC) routes by `(channel_id, stream_type)` regardless of convention. The convention choice — *how stream_type values are assigned to roles* — is a documentation/ergonomics choice, not a mechanism choice. Both conventions work; the question is which is cleaner. ### ADR-071's current convention: mod-3 groups | Group | stream_type | direction | purpose | |-------|-------------|-----------|---------| | Data | 0 | write | in (stdin) | | | 1 | read | out (stdout) | | | 2 | read | err (stderr) | | Control | 3 | write | ctrl_in | | | 4 | read | ctrl_out | | | 5 | read | ctrl_err (optional) | | Future | 6/7/8, 9/10/11, ... | write/read/read | next groups | Formula: `stream_type % 3 == 0` → write, `% 3 == 1` → read, `% 3 == 2` → diagnostic read (err). **The flaw: stderr is structural.** The `+2 = err` slot bakes a *unidirectional* role (stderr is server→client only) into a *bidirectional* group structure (in/out/err). Every group either allocates an err slot it may not use (tunnel has no err) or skips the group structure. The group concept (data vs control) is real, but the 3-slot shape is asymmetric — bidirectional things are two halves, but the group has three slots, one of which is a unidirectional leaf. ### Proposed convention: mod-2/mod-4 by instance An **instance** is a self-contained bidirectional unit, addressed as a contiguous block of `stream_type` values within a channel. **No control channel** — mod 2: | Instance | in (write) | out (read) | |----------|-----------|-----------| | 0 | 0 | 1 | | 1 | 2 | 3 | | 2 | 4 | 5 | | ... | | | | 127 | 254 | 255 | 128 instances per channel. Instance K uses `stream_type = 2K` (in) and `2K+1` (out). **With control channel** — mod 4: | Instance | in (write) | out (read) | ctrl_in (write) | ctrl_out (read) | |----------|-----------|-----------|-----------------|-----------------| | 0 | 0 | 1 | 2 | 3 | | 1 | 4 | 5 | 6 | 7 | | 2 | 8 | 9 | 10 | 11 | | ... | | | | | | 63 | 252 | 253 | 254 | 255 | 64 instances per channel. Instance K uses `4K` (in), `4K+1` (out), `4K+2` (ctrl_in), `4K+3` (ctrl_out). **The `stream_type` space is the instance address space, not a role space.** A handler talks about "instance K of my channel" as a contiguous block of stream_types. This is richer than ADR-071's "data group / control group" framing — the instance is the addressing unit, and a handler can have N independent bidirectional sub-streams on one channel, each with or without its own control. **Combined address space:** `channel_id × instance`. Channel 0 is hardcoded `alknet/call` (ADR-072), so channels 1..255 are dynamic — ~255 × 128 (no control) or ~255 × 64 (with control) logical sub-streams per channels connection. Huge; the load-bearing count for the recursive-multiplexing property below. ### Why mod 2 wins 1. **Uniformity.** Every bidirectional thing is exactly two stream_types (write, read). Control is a pair; io is a pair; future sub-streams are pairs. No "err is a third half" asymmetry. 2. **The "unused write half" wart is trivially clean** (verified in the existing POC code — see "Stream_type split/recombine" above). An unused `stream_type` is an idle mpsc channel: no dangling sender, no premature EOF (EOF fires on sender drop or zero-length sentinel), no flow-control weirdness (independent bounded buffers). The lenient unknown-`stream_type` path (`demux.rs:161-174`) means even a stray write to an unallocated `stream_type` is dropped with a counter bump, not a panic. 3. **Stderr is just another instance.** PTY mode (stderr merged into stdout server-side) declares instance 0 only. Pipe mode (separate stderr) declares instance 0 (io) + instance 1 (stderr-as-bidirectional, write half unused). The unused write half costs one idle mpsc channel; cheap. No structural asymmetry. 4. **The demux/mux doesn't care.** The convention is documentation; the mechanism routes by `(channel_id, stream_type)` regardless. Both conventions work; mod 2 is the cleaner documentation. **The mod-2-vs-mod-3 question is settled by the existing POC evidence.** No new POC needed. The mechanism supports both; mod 2 wins on uniformity, and the "unused write half" property that made mod 2 look like a wart is trivially clean in the actual code. --- ## The TTY control channel flaw (separate, already specified) `crates/alknet-tty/src/wire.rs:13,32-33`: ``` STREAM_CONTROL: u8 = 3 // "bidirectional, JSON control message" InvalidStreamType > 3 ``` One `stream_type` both sides write to — the "control isn't actually bidirectional" flaw. ADR-071 §stream_type decomposition already fixes this at the wire-format level (stream_types 3 = ctrl_in, 4 = ctrl_out); ADR-077 amends ADR-052 to add stream_type 4. The TTY code hasn't been updated — it still has the old "bidirectional 3" comment and the `> 3` bound. **This is an implementation-lag, not a design question.** The fix is specified; the code needs to catch up. The mod-2/mod-4 instance framing subsumes this fix: control becomes instance 0's ctrl_in/ctrl_out pair (types 2/3 under mod 4), not a standalone "bidirectional" stream_type. --- ## TTY/channels convergence Channels was written after TTY as a natural extension: TTY's 5-byte header + `channel_id:u32` prefix = the 9-byte channels header. The convergence has two parts. ### Semantic convergence (independent of format) Regardless of format choice, TTY and channels present the same shape to handlers: a set of unidirectional sub-streams declared per-channel at `channel/open` time, paired into bidirectional instances where the handler wants a joined `BiStream`. `into_sub_streams()` (ADR-074) returns the declared stream_types as `Vec<(u8, SubStreamHandle)>` with `SubStreamHandle::Send` (write half) or `Recv` (read half). The handler joins the pairs it wants via `tokio::io::join`. The channels layer exposes the leaves; the handler composes them. ADR-074's two paths (`accept_bi` for the joined 0/1 pair, `into_sub_streams` for the typed leaves) become the same data, presented differently. `accept_bi` is a convenience that joins 0/1 for the common case (tunnel, SSH, echo, HTTP); `into_sub_streams` is the primary accessor (the unidirectional leaves). ### Format convergence (the open question) **Option A (current, ADR-077): two formats coexist.** TTY-direct keeps the 5-byte format; TTY-inside-channels uses 9-byte. The 4-byte `channel_id` overhead is paid only inside channels. **Option B (retire the 5-byte format): TTY-direct uses 9-byte.** One demux implementation, one set of stream_type semantics, one crate. Sub-options for the `channel_id`: - **B1:** TTY-direct is a channels connection with `channel_id = 0` = `alknet/tty` (breaks ADR-072's "channel 0 is call" rule). - **B2:** TTY-direct is a channels connection with `channel_id = 0` = `alknet/call` (unused, no `channel/open` issued) + `channel_id = 1` = `alknet/tty` (pre-allocated). Pays one unused channel for rule consistency. Channel 0 is *always* call, even when unused. **Option B is the bigger call.** It's a wire-format change with backward-compat implications. It's the one open question that benefits from a POC — see below. --- ## The recursive multiplexing property A channels connection with N channels, each with up to 128 (or 64) instances, has N×128 (or N×64) logical sub-streams. An instance can itself be a channels connection (recursive composition, ADR-074 — a channel with ALPN `alknet/channels` inside `alknet/channels`). The "channel within a channel within a channel" shape is unbounded, and the mod-2/mod-4 instance framing makes each level uniform — every level is "pairs of stream_types (or 4-tuples with control), declared per-channel, addressed by instance." This is a property, not a feature. The primary use case is one level of multiplexing. Recorded here because the instance framing makes it cleaner than ADR-071's group framing did — the instance is the recursive unit; the group was not. --- ## POC candidates ### POC 1: stderr split/recombine — **already answered, no POC needed** **Original framing:** "If stderr is its own instance (mod 2, write half unused), does the unused write half cause problems — dangling sender, premature EOF, flow-control weirdness?" **Answer (verified in existing POC code):** No. The demux/mux is per-`stream_type` independent — there is no pairing assumption in the mechanism. An unused `stream_type` is an idle mpsc channel. The lenient unknown-`stream_type` path drops stray writes with a counter bump. The 28-test channels POC already validated per-`stream_type` independence (`demux_three_concurrent_channels_no_cross_contamination`, `demux_unknown_channel_drops_lenient`, `recv_dropped_sender_is_eof`). **Status:** Confirmatory, not exploratory. The load-bearing property is already proven. The mod-2 framing is trivially clean. No new POC. ### POC 2: TTY-direct-as-channels — the one open question **Question:** Is Option B1 (TTY-direct = channels with `channel_id = 0` = `alknet/tty`, breaking ADR-072) or Option B2 (channel 0 = call unused, channel 1 = TTY) feasible and clean? Or is the 4-byte `channel_id` overhead per chunk in TTY-direct mode a real concern for terminal I/O? And is backward-compat with existing TTY-direct deployments a real constraint, or are there no existing deployments to break? **Why it matters:** If Option B is clean and backward-compat is non-binding, the 5-byte format retires, the demux implementation unifies, and the TTY crate's direct mode becomes a thin wrapper. If backward-compat is binding or the overhead is a concern, Option A (two formats coexist per ADR-077) stands. **Scope:** ~200 lines. A POC that runs the TTY adapter over a channels-format demux with `channel_id = 0` (B1) and `channel_id = 1` (B2), using the existing TTY `TtyBackend` (the local pipe backend). Verify the `pty.rs` / `pipe.rs` tests still pass in shape. Measure chunk overhead if relevant. **Output:** A `docs/research/alknet-channels/poc-tty-direct-as-channels.md` summary. Recommends Option A, B1, or B2. If B, becomes the input to ADR-094 (retire the 5-byte format). If A, ADR-077 stands and the 5-byte format is kept. ### POC 3: recursive composition — low leverage, deferred **Question:** Does `alknet/channels` inside `alknet/channels` work end-to-end? The abstraction permits it (ADR-074); the POC didn't validate it (`poc-summary.md` §"What the POC Does NOT Validate" #5). **Why it matters:** Low leverage — the instance framing makes it cleaner, but the primary use case is one level. Recursive composition is a property, not a feature. **Status:** Deferred. Not a POC for this round. --- ## What goes where (ADR plan) | ADR | Scope | Status | |-----|-------|--------| | **ADR-092** | Transport leaf: `BiStream` as the handler leaf; `accept_bi` returns `BiStream`; `from_stream` removed; `from_bidi` is the only public stream constructor. | **Drafted, pushed** (`f8d4650`, `528cfa0`). Load-bearing, separable. | | **ADR-093** | Multiplexing: mod-2/mod-4 instance framing; per-channel stream_type declaration; `into_sub_streams()` as the primary accessor; demux convention-agnostic. Amends ADR-071 (stream_type decomposition), ADR-074 (`into_sub_streams`), ADR-077 (TTY uses mod 2; control channel fixed). | **Ready to draft.** The mod-2-vs-mod-3 question is settled by existing POC evidence; the control channel fix is already specified in ADR-077; the instance framing is the cleaner documentation. | | **ADR-094** | Format convergence: retire the 5-byte TTY-direct format in favor of 9-byte channels format. Option A (keep 5-byte), B1 (channel 0 = TTY), or B2 (channel 0 = call unused, channel 1 = TTY). Backward-compat analysis. | **Pre-ADR**, drafting after POC 2 recommends an Option. | --- ## Open questions - **5-byte format retirement.** POC 2 resolves. Default assumption: Option A (two formats coexist, ADR-077 stands) is safer; Option B (retire 5-byte) is cleaner but a wire-format change. POC validates feasibility and overhead. - **`channel 0 is always call` (ADR-072) under TTY-direct-as-channels.** If POC 2 recommends B1 (TTY-direct channel 0 = `alknet/tty`), ADR-072 needs an exception or amendment. If B2 (channel 0 = call unused, channel 1 = TTY), ADR-072 stands. If A (keep 5-byte), no change. - **Recursive composition.** Deferred. The instance framing makes it cleaner, but it's not a goal. POC 3 is low leverage. --- ## References - ADR-092: `BiStream` as the handler leaf (the transport-leaf layer, settled; this doc is the layer above it) - ADR-071: channels wire format (the mod-3 stream_type decomposition ADR-093 amends) - ADR-074: `ChannelBidiStreamSource` / `into_sub_streams` (the per-channel sub-stream accessor ADR-093 amends) - ADR-077: TTY inside channels (the two-mode TTY design ADR-093 amends; the 5-byte format scoping ADR-094 may retire) - ADR-072: channel 0 pre-negotiated as `alknet/call` (the hardcoded channel_id constraint) - ADR-073: channel lifecycle operations (the `stream_types` field declared at `channel/open` time — the per-channel declaration that makes the demux convention-agnostic) - `docs/research/alknet-channels/poc-summary.md` — the channels POC (28 tests) that validated per-`stream_type` independence - `/workspace/alknet-channels-poc/src/demux.rs:91-109, 161-181` — the demux's per-`stream_type` routing (no pairing assumption) - `/workspace/alknet-channels-poc/src/mux.rs:58-63, 152-177` — the mux's per-`(channel_id, stream_type)` independent pumps - `crates/alknet-tty/src/wire.rs:13,32-33` — the `STREAM_CONTROL = 3` "bidirectional" flaw (implementation lag; fix specified in ADR-077, subsumed by the mod-4 instance framing) - `crates/alknet-tty-local/tests/pipe.rs:363 separate_stderr` — the test that proves separate stderr is a real feature, not vestigial (resolves under mod 2 as instance 1 with unused write half)