alktunnels/docs/research/phase-0-findings.md

status, last_updated

status	last_updated
draft	2026-09-05

alktunnels — Phase 0 Research Findings

This document captures Phase 0 (Exploration) findings and open design questions for the alktunnels crate. The objective of Phase 0 per docs/sdd_process.md is: "Capture vision and guiding principles; research options; validate approaches; converge on a recommended approach." It is the input to Phase 1 (Architecture), where the Architect will produce docs/architecture/ specs, ADRs, and open questions.

Drafted 2026-09-05, emerging from the initial setup discussion. The crate is the sibling of alktty (alk/tty — terminal sessions) on the alkcall substrate: where alktty multiplexes one service with a fixed five-stream channel structure, alktunnels generalizes the tunnel handler shape to arbitrary bidirectional tunnels in the ssh -L / ssh -D sense — TCP, UDP, unix sockets, and other stream or datagram substrates.

The 2026-09-05 first revision adds the tun2proxy UDP gateway prior art (OQ-TN-01, OQ-TN-02, OQ-TN-07, OQ-TN-10) and the transport story clarification (TCP vs QUIC at the channels layer, invisible to this crate — §What is already settled). This prior art was discussed with the POC agent around the alknet-channels POC but never documented; it is captured here so it survives into the spec.

What is already settled

The foundation is POC-validated and ADR-pinned; this crate is not starting from zero. It inherits:

• The demux→Connection→handler→mux path — validated by the alknet-channels POC (Target 3), now production alkcall channels. The tunnel payload is raw bytes inside a channels data channel; channels strips its 8-byte header transparently (alknet ADR-093 / alkcall ADR-035).
• The two-pump handler shape — one pump per direction, each pump MUST shut down the opposite sink on completion (try_join! alone deadlocks; alknet ADR-078). POC-validated with a 1 MiB backpressure test. This crate is the second two-pump consumer the ADR deferred helper extraction for (the first was the POC's tunnel handler; SSH direct-tcpip would be a later third).
• The producer/consumer model — producer registers openable channels via ChannelCore::register_openable (authorization for free via AccessControl); consumer opens tunnel channels via ChannelClient (alkcall ADR-037, ADR-043). Connection direction is independent of tunnel direction.
• The backend inversion point pattern — substrate-specific types (TcpStream, UdpSocket, unix sockets) confined to feature-gated backend modules, injected at the assembly layer, never imported from the shared/producer/consumer modules (alktty TtyBackend precedent).
• The wasm-clean default crate — protocol-only code compiles to wasm32-unknown-unknown; socket/platform I/O is feature-gated (alktty precedent).
• The relay story — tunnels traverse alkcall hub relays transparently via byte-for-byte data-channel forwarding with ID rewrite (alkcall ADR-042). No tunnel-specific relay work.
• The transport story — the underlying channels transport is TCP or QUIC (QUIC preferred) but that is the alkcall layer's concern, not this crate's: the tunnel protocol sees a BiStream and is transport-agnostic like the sibling crates. UDP tunneling (below) rides the same chunk stream; the stream-vs-datagram question is about what the tunnel protocol frames inside the channel, not about the transport.

Prior art: tun2proxy UDP gateway (OQ-TN-01, OQ-TN-02, OQ-TN-07, OQ-TN-10)

/workspace/tun2proxy/src/udpgw.rs implements a UDP gateway over a TCP stream — structurally the same problem this crate faces for UDP tunnels over channels data channels. Discussed with the alknet-channels POC agent as the example of "UDP over a stream substrate," but never documented. Key mechanics, all of which generalize:

• Per-datagram length framing over the stream — the packet format is LEN(u16 BE) | FLAGS(u8) | CONN_ID(u16) | [SOCKS5 address] | DATA (udpgw.rs:82-88). Boundary preservation is re-added by the protocol, not by the substrate: exactly the "length-prefix each datagram inside the channel" half-answer in OQ-TN-02, proven in production.
• SOCKS5 address format travels per data packet — ATYP (0x01 IPv4 / 0x03 domain / 0x04 IPv6) + variable address + port (udpgw.rs:68-76). This is concrete prior art for OQ-TN-01's addressing: a scheme-tagged addressing encoding with v4/v6/domain coverage already standardized. Note the asymmetry with TCP tunnels: for UDP, the remote endpoint is per-datagram, not per-channel.
• One stream carries many UDP flows — CONN_ID(u16) multiplexes associations over a single gateway connection (udpgw.rs:66), with keepalive (0x01) and error (0x20) flag packets as the only non-data frame types (udpgw.rs:21-26). This is the "one channel = one association, per-endpoint multiplexing inside" half-answer in OQ-TN-02, with the refinement that the per-endpoint multiplexing key (CONN_ID) is protocol-level, allocated by the client, u16.
• Flow lifecycle is packet-level — udp_timeout idle expiry, keepalive_time heartbeats on idle connections (UDPGW_KEEPALIVE_TIME = 30s, udpgw.rs:16), and an MTU cap (parse_udp_response rejects data.len() > udp_mtu, udpgw.rs:527). Also UDPGW_MAX_CONNECTIONS = 5 pooled gateway connections above the packet layer — a throughput choice, not a protocol requirement; channels gives us N channels already.
• Implications for alktunnels:
  • UDP boundary preservation over the chunk stream is validated prior art, not speculation — raises confidence in the OQ-TN-02 half-answer considerably.
  • The frame-type set (DATA/KEEPALIVE/ERR) is a useful minimal vocabulary — it maps onto OQ-TN-09's establishment/error frame question (tun2proxy uses flag packets, alktty uses typed control chunks; both are self-contained frames inside the data stream).
  • Per-datagram addressing (SOCKS5-style) vs per-channel addressing (fixed target at open) is a real fork for the params design: TCP tunnels fix the target at open; UDP associations may either fix one endpoint at open or carry per-datagram addresses like udpgw. This interaction is unresolved and feeds OQ-TN-01 + OQ-TN-07.
  • NAT/keepalive concerns partially disappear on channels: the underlying transport (QUIC/TCP) handles connection keepalive, and channel liveness is the channels layer's concern. The tunnel protocol likely needs only flow-level idle expiry, not transport-level keepalive packets — TBD in the spec.

Open Questions

These are the design questions Phase 0 must resolve (or explicitly defer) before the architecture spec. They are numbered OQ-TN-01.. so they can be referenced, tracked, and promoted into docs/architecture/open-questions.md in Phase 1. Half-answers and hunches are marked as such — the point of this document is to hold them without forcing premature decisions.

OQ-TN-01: Target addressing format

What does the tunnel params on channel/open look like? alknet ADR-071 §ALPN table noted alknet/tunnel as [0, 1] data in/out only, but the addressing scheme was never decided. It must cover at minimum:

• TCP dial (host:port)
• UDP (associate-style or endpoint-style — see OQ-TN-02)
• Unix domain sockets (path)
• Direction (who dials the target — see OQ-TN-03)
• Bind/listen vs dial semantics (see OQ-TN-04)

Considerations:

• params is ALPN-specific JSON, interpreted by the open handler, not by the channels layer (alknet ADR-075 / alkcall ADR-039). alktty's precedent is the NegotiateRequest shape — a self-contained JSON object carried in the open op.
• The addressing string is wire-stable once a consumer exists (one-way door). It must be substrate-extensible without format changes (a new substrate should be an additive scheme value, not a v2 format).
• Prior art to survey: SSH forwarding models (direct-tcpip, forwarded-tcpip, direct-udpip in some implementations), SOCKS5 addressing (ATYP + addr + port — supports v4/v6/domain + UDP associate), iroh/tun2proxy target encoding, quinn-proxy-poc.

Status: open — research needed. Half-answer (hunch): a scheme-tagged JSON object rather than a URL-ish string, so params stay typed and extensible; exact shape TBD. Strengthened 2026-09-05: the tun2proxy UDP gateway (§Prior art) validates SOCKS5 ATYP addressing (v4/v6/domain) as in-band prior art, and surfaces a fork — for UDP, remote addressing is per-datagram (SOCKS5-style) rather than fixed-at-open like TCP. The params design must account for both modes.

OQ-TN-02: Datagram substrates (UDP) — boundary preservation

Does a UDP tunnel preserve datagram boundaries end-to-end, or does the tunnel present a byte-stream abstraction to the consumer (boundaries lost, re-chunked arbitrarily)?

• Channels is a chunk stream with bounded buffers; the zero-length chunk is the EOF sentinel — datagram boundaries are not preserved by the substrate (alknet ADR-071/093; the POC only exercised TCP).
• SSH's -D UDP associate tunnels UDP as a stream with per-datagram framing re-added by the tunnel protocol (e.g. SOCKS5 UDP over TCP). russh/openssh do this differently — survey needed.
• iroh and quinn-proxy-poc have native datagram transports; tun2proxy has a full UDP-over-TCP model worth reading.
• Boundary preservation is a wire-format decision (per-datagram length framing inside the BiStream) and would need an ADR + possibly a BAST document (AGENTS.md convention 12). Boundary loss is cheaper but changes what protocols can ride the tunnel (DNS? QUIC? game traffic?).
• Datagrams also raise multiplexing questions TCP does not: one UDP "association" carries many remote endpoints — does one tunnel channel carry one endpoint or many, and how are per-endpoint replies routed?

Status: open — survey mostly resolved by tun2proxy prior art (§Prior art): per-datagram length framing over the stream is production-proven (LEN | FLAGS | CONN_ID | [addr] | DATA), one stream carries many UDP flows via a protocol-level CONN_ID, and flow lifecycle (idle timeout + keepalive) is packet-level. Remaining: whether alktunnels fixes the UDP endpoint at open (per-channel, TCP- like) or carries per-datagram addresses (udpgw-like), and whether a u16 conn-id vocabulary is right for channels (vs the channel ID itself doing the demux and one channel per UDP flow). A targeted POC (OQ-TN-10 #1) is likely still +EV for the chosen shape.

OQ-TN-03: Direction semantics (-L / -R / dynamic)

SSH has three forwarding flavors; the crate must model them without "server/client" framing:

• -L (local forward): consumer dials a local port; producer dials the target. Channels flows consumer→producer; target dial happens on the producer side. This is the POC's shape.
• -R (remote forward): producer (or a third party) listens; the consumer's side dials or accepts incoming connections and asks the other side to carry them. Channels flows producer→consumer.
• -D (dynamic/SOCKS): one side runs a SOCKS5 server; the target is chosen per-connection by the client. Addressing arrives per-channel, not per-tunnel-registration.

Both sides can be producer and consumer simultaneously (alkcall ADR-022/037 direction semantics), so the model must not bake direction into the connection. The open questions:

• Is direction a field in params, or two distinct open-handler shapes / ALPNs?
• How does -R register availability (the side that will carry traffic advertises listen targets)? Does it interact with channel/open at all, or is it a call-level operation ("please open a tunnel channel to me when a local accept happens")?
• Dynamic (-D) may not be a tunnel concern at all — it may compose as "SOCKS5 server implemented over alktunnels dial primitives" in a separate crate. Keep or cut for v1?

Status: open — needs architecture decision. Half-answer (hunch): -L is the channel/open handler; -R needs a small advertisement/lifecycle surface; -D composes on top and is out of scope for the base crate.

OQ-TN-04: No forced local binding

A tunnel must not require the producer (or consumer) to bind a local port. The POC's shape dialed a target from the handler; binding is optional and belongs to the caller (assembly layer), not the protocol crate. The API surface must support:

• Dial flows with no local bind (POC shape) — covered.
• Listen flows where the binding happens on one side only.
• Unbound/abstract flows (e.g. unix socketpair-style, stdio bridges, in-process pipes) where neither side binds.

The protocol layer must express "carry bytes between this target and this channel" without assuming either endpoint is a bound socket. Substrate modules (behind feature flags) own actual bind() calls; the protocol owns bookkeeping only.

Status: open — mostly a spec-level requirement to encode in the architecture docs and API shapes rather than a research question. Half- answer: already agreed as a requirement (AGENTS.md convention 9); what's missing is the concrete API surface (who calls what to start a tunnel in each mode).

OQ-TN-05: Backend inversion point — is there a TunnelBackend trait?

alktty has TtyBackend because backends (local PTY, docker, SSH) produce handles and the adapter pumps them. For tunnels, the producer side's substrate action is narrower — dial a target, or accept on a listener — so the question:

• Is a TunnelBackend-style trait needed at all, or is the two-pump handler + feature-gated substrate modules (dial/listen helpers) the whole story, with the assembly layer wiring substrate streams directly?
• If a trait: what is the handle type? A tunnel "handle" is just an AsyncRead + AsyncWrite stream (or a datagram endpoint) — much thinner than TtyHandle's stdin/stdout/stderr/exit-code quadruple. The trait may collapse to "produce a boxed stream for this target" plus a listener variant.
• Backpressure/limits come from channels (AGENTS.md convention 10); the backend trait must not add a second layer of them.

Status: open — needs a survey of what backends would actually implement (local TCP? docker exec? ssh -w?) before deciding trait vs no-trait. Half-answer (hunch): a thin trait (or just a fn alias) for "obtain a bidirectional substrate stream for a target," possibly no trait at all if the only meaningful backends are local sockets — decide after surveying candidate backends.

OQ-TN-06: The two-pump helper — extract now?

alknet ADR-078 deferred helper extraction until a second two-pump consumer exists ("a genuine deferral... the contract is decided (shutdown-on- completion), only the extraction is deferred"). This crate is that second consumer (POC tunnel was the first; SSH direct-tcpip would be a third).

• Does the helper live here (as a pub utility other handler crates can use), or upstream in alkcall (which already owns core types)?
• Shape: pump_bidi(recv, send) -> (Future, Future) returning both pumps with the shutdown-on-completion wired in? Or a join_two_pumps(a, b) combinator?
• alknet ADR-057 (two-pump helper extraction OQ) noted the helper from one consumer would bake in a wrong shape; with two consumers the shapes should be compared before extraction.

Status: open — decide when the first real tunnel handler is written; not a blocker for the spec. Half-answer: the helper probably belongs upstream (alkcall, near the channels-adapter handler-integration conventions) but only if the two shapes genuinely converge.

OQ-TN-07: ALPN strategy

This crate owns the alk/tunnel-family ALPN(s). alkcall ADR-004: one ALPN per protocol; alk/ prefix. If stream (TCP/unix) and datagram (UDP) tunnels get distinct ALPNs, the split must be decided before the first consumer — ALPN strings are wire-stable once published.

• Option A: single alk/tunnel ALPN; substrate is a params field (and datagram framing, if any, is self-describing inside the channel).
• Option B: alk/tunnel (stream) + alk/tunnel-dgram (datagram), so the wire framing differs per ALPN cleanly.
• Channels' params is ALPN-specific, and the open-handler registry dispatches per ALPN — both options are cheap mechanically; the cost is consumer-side API bifurcation (two session types vs one with a substrate enum).

Status: open — needs the OQ-TN-02 outcome first (if datagrams need different framing, option B gets stronger). Note from the tun2proxy prior art (§Prior art): udpgw runs its packet framing over a plain TCP stream — one framing covers both the stream and datagram cases there. If alktunnels follows the same shape (datagram framing self-describing inside the channel), option A (single alk/tunnel ALPN) stays viable even with UDP support; option B remains cleaner if the datagram channel needs structurally different framing from the first chunk on.

OQ-TN-08: Access control and ownership scope

Tunnels reach local networks — the open gate is the security boundary. Shape follows alktty: TUNNEL_OPEN_SCOPE scope-gate, and the channels path gets AccessControl wiring for free via ChannelCore::register_openable. Open sub-questions:

• Should ownership (OwnershipProvider.owns(...)) be consulted for tunnel targets, and what is the resource identity of a tunnel target (a host:port? a registered tunnel name?), given targets may be arbitrary strings and wildcard targets (0.0.0.0/0-style egress) may be intentionally allowed for some identities?
• Is there a target-allowlist concept (per-identity reachable target sets), and does it live in AccessControl or in the open handler's params validation?

Status: open — needs alkcall ADR-050 review + a survey of how alktty scoped its gate. Half-answer (hunch): scope-gate for the open plus an open-handler-level target policy hook; ownership for registered/listened tunnels (which are persistent resources), not for ephemeral dials.

OQ-TN-09: Lifecycle, teardown, and error reporting

The two-pump shape gives byte-level teardown for free (EOF sentinels; channels drops per-channel senders on transport EOF — alknet ADR-078, POC issue #6). What's missing is the error/level above bytes:

• How does a failed target dial reach the consumer (e.g. "connection refused to 10.0.0.5:80")? Is there a structured error frame in the channel before close, a channel/close with reason, or call-level error on the open op?
• Is there a "tunnel established/failed" ack before byte pumping starts (alktty has the negotiation frame; the POC's tunnel handler had nothing — it dialed and pumped)?
• Half-open semantics: one direction EOFs, the other keeps pumping (standard two-pump behavior) — is that always desired, or does the consumer need a "close both" control?

Status: open — needs a wire-format decision (ADR) if an error frame is added. Half-answer (hunch): a self-contained control frame (alktty ADR-006 shape) carrying an establishment result/error, sent before any data chunk; dial errors are tunnel-closing (the whole channel dies), whereas byte-level EOFs stay per-direction.

OQ-TN-10: POC scope for what remains unvalidated

The alknet-channels POC validated TCP only. Candidate targeted POCs Phase 0 may need (in rough priority order, per the SDD process's "validate promising approaches"):

1. UDP tunnel POC — boundary-preserving length framing over a channels channel, per-endpoint multiplexing inside one association, backpressure behavior. Partially derisked by the tun2proxy prior art (§Prior art) — the POC now mainly validates channels-layer fit: chunk-size vs datagram-size interaction, MTU cap against the channels bounded buffers, idle-expiry mapping, and the chosen endpoint-addressing shape. Derisks OQ-TN-02 (and OQ-TN-07's option B).
2. Reverse-flow POC — -R-style: the accept side listens, the far side carries. Derisks OQ-TN-03's advertisement/lifecycle shape.
3. Unix socket + stdio bridge POC — cheap; validates "substrate agnostic" beyond IP substrates.
4. Two-pump helper extraction spike — OQ-TN-06, only after 1–3.

POCs live in .worktrees/research/<task-id>/ per the SDD process, or as standalone crates (/workspace/alknet-channels-poc precedent).

Status: open — pick 1 (and probably 2) after the research pass; 3 is cheap enough to fold into whichever POC runs first.

Survey / prior-art list

Candidate reading for the research specialist (to be expanded):

• SSH channel/forwarding model: RFC 4254 §7 (direct-tcpip / forwarded-tcpip), OpenSSH -L/-R/-D semantics, russh's ChannelOpen framing (russh is already in /workspace/russh).
• SOCKS5 (RFC 1928): addressing (ATYP), UDP ASSOCIATE framing, per-endpoint multiplexing — the closest standardized "arbitrary tunnel + UDP" model.
• tun2proxy (/workspace/tun2proxy, src/udpgw.rs): UDP gateway over TCP — per-datagram length framing, SOCKS5 per-datagram addressing, CONN_ID flow multiplexing, keepalive/ERR flag packets, MTU cap, idle expiry. Analyzed in §Prior art. Its socks.rs / proxy_handler.rs are also relevant for the -D (dynamic/SOCKS) composition question (OQ-TN-03).
• quinn-proxy-poc (/workspace/quinn-proxy-poc) and iroh (/workspace/iroh): datagram-native transports; how they model per-endpoint flows.
• alknet docs: ADR-071 §ALPN table (alknet/tunnel row), ADR-078, docs/architecture/crates/channels/channel-operations.md (params for alknet/tunnel is "the target resource"), and the hub-relay interaction (ADR-042/079).
• alktty: NegotiateRequest shape (self-contained negotiation precedent), TtyBackend inversion point, TTY_OPEN_SCOPE access gate.

Convergence checklist (what Phase 0 must produce)

• Survey notes: SSH/SOCKS5/tun2proxy addressing + UDP framing (OQ-TN-01, OQ-TN-02) — tun2proxy UDP gateway done (§Prior art); SSH/SOCKS5 addressing survey still open
• Recommendation: addressing format sketch (OQ-TN-01) — SOCKS5 ATYP validated as in-band encoding prior art; per-channel vs per-datagram fork unresolved
• Recommendation: datagram strategy (OQ-TN-02) + ALPN strategy dependent on it (OQ-TN-07) — framing mechanics de-risked by tun2proxy prior art; endpoint-at-open vs per-datagram addressing fork remains
• Recommendation: direction model (-L/-R/-D) (OQ-TN-03) + API surface sketch satisfying no-forced-binding (OQ-TN-04)
• Decision input: backend trait vs no-trait (OQ-TN-05)
• Targeted POC(s) run + summary (OQ-TN-10) — UDP first, reverse flow second
• Open questions promoted to Phase 1 docs/architecture/open-questions.md with statuses