diff --git a/docs/research/phase-0-findings.md b/docs/research/phase-0-findings.md new file mode 100644 index 0000000..de93501 --- /dev/null +++ b/docs/research/phase-0-findings.md @@ -0,0 +1,344 @@ +--- +status: draft +last_updated: 2026-09-05 +--- + +# alktunnels — Phase 0 Research Findings + +This document captures Phase 0 (Exploration) findings and open design +questions for the `alktunnels` crate. The objective of Phase 0 per +`docs/sdd_process.md` is: *"Capture vision and guiding principles; research +options; validate approaches; converge on a recommended approach."* It is the +input to Phase 1 (Architecture), where the Architect will produce +`docs/architecture/` specs, ADRs, and open questions. + +Drafted 2026-09-05, emerging from the initial setup discussion. The crate is +the sibling of alktty (`alk/tty` — terminal sessions) on the alkcall +substrate: where alktty multiplexes one service with a fixed five-stream +channel structure, alktunnels generalizes the tunnel handler shape to +arbitrary bidirectional tunnels in the `ssh -L` / `ssh -D` sense — TCP, UDP, +unix sockets, and other stream or datagram substrates. + +## What is already settled + +The foundation is POC-validated and ADR-pinned; this crate is not starting +from zero. It inherits: + +- **The demux→Connection→handler→mux path** — validated by the + alknet-channels POC (Target 3), now production alkcall channels. The + tunnel payload is raw bytes inside a channels data channel; channels + strips its 8-byte header transparently (alknet ADR-093 / alkcall + ADR-035). +- **The two-pump handler shape** — one pump per direction, each pump MUST + shut down the opposite sink on completion (`try_join!` alone deadlocks; + alknet ADR-078). POC-validated with a 1 MiB backpressure test. This + crate is the *second* two-pump consumer the ADR deferred helper + extraction for (the first was the POC's tunnel handler; SSH + `direct-tcpip` would be a later third). +- **The producer/consumer model** — producer registers openable channels + via `ChannelCore::register_openable` (authorization for free via + `AccessControl`); consumer opens tunnel channels via `ChannelClient` + (alkcall ADR-037, ADR-043). Connection direction is independent of + tunnel direction. +- **The backend inversion point pattern** — substrate-specific types + (`TcpStream`, `UdpSocket`, unix sockets) confined to feature-gated + backend modules, injected at the assembly layer, never imported from + the shared/producer/consumer modules (alktty `TtyBackend` precedent). +- **The wasm-clean default crate** — protocol-only code compiles to + `wasm32-unknown-unknown`; socket/platform I/O is feature-gated + (alktty precedent). +- **The relay story** — tunnels traverse alkcall hub relays + transparently via byte-for-byte data-channel forwarding with ID + rewrite (alkcall ADR-042). No tunnel-specific relay work. + +## Open Questions + +These are the design questions Phase 0 must resolve (or explicitly defer) +before the architecture spec. They are numbered OQ-TN-01.. so they can be +referenced, tracked, and promoted into `docs/architecture/open-questions.md` +in Phase 1. Half-answers and hunches are marked as such — the point of this +document is to hold them without forcing premature decisions. + +### OQ-TN-01: Target addressing format + +What does the tunnel `params` on `channel/open` look like? alknet ADR-071 +§ALPN table noted `alknet/tunnel` as `[0, 1]` data in/out only, but the +addressing scheme was never decided. It must cover at minimum: + +- TCP dial (`host:port`) +- UDP (associate-style or endpoint-style — see OQ-TN-02) +- Unix domain sockets (path) +- Direction (who dials the target — see OQ-TN-03) +- Bind/listen vs dial semantics (see OQ-TN-04) + +Considerations: + +- `params` is ALPN-specific JSON, interpreted by the open handler, not by + the channels layer (alknet ADR-075 / alkcall ADR-039). alktty's + precedent is the `NegotiateRequest` shape — a self-contained JSON + object carried in the open op. +- The addressing string is wire-stable once a consumer exists (one-way + door). It must be substrate-extensible without format changes (a new + substrate should be an additive `scheme` value, not a v2 format). +- Prior art to survey: SSH forwarding models (`direct-tcpip`, + `forwarded-tcpip`, `direct-udpip` in some implementations), SOCKS5 + addressing (ATYP + addr + port — supports v4/v6/domain + UDP associate), + iroh/tun2proxy target encoding, quinn-proxy-poc. + +**Status:** open — research needed. Half-answer (hunch): a scheme-tagged +JSON object rather than a URL-ish string, so params stay typed and +extensible; exact shape TBD. + +### OQ-TN-02: Datagram substrates (UDP) — boundary preservation + +Does a UDP tunnel preserve datagram boundaries end-to-end, or does the +tunnel present a byte-stream abstraction to the consumer (boundaries lost, +re-chunked arbitrarily)? + +- Channels is a chunk stream with bounded buffers; the zero-length chunk + is the EOF sentinel — datagram boundaries are *not* preserved by the + substrate (alknet ADR-071/093; the POC only exercised TCP). +- SSH's `-D` UDP associate tunnels UDP as a stream with per-datagram + framing re-added by the tunnel protocol (e.g. SOCKS5 UDP over TCP). + russh/openssh do this differently — survey needed. +- iroh and quinn-proxy-poc have native datagram transports; tun2proxy has + a full UDP-over-TCP model worth reading. +- Boundary preservation is a wire-format decision (per-datagram length + framing inside the `BiStream`) and would need an ADR + possibly a BAST + document (AGENTS.md convention 12). Boundary loss is cheaper but + changes what protocols can ride the tunnel (DNS? QUIC? game traffic?). +- Datagrams also raise multiplexing questions TCP does not: one UDP + "association" carries many remote endpoints — does one tunnel channel + carry one endpoint or many, and how are per-endpoint replies routed? + +**Status:** open — research needed (survey SSH/russh/SOCKS5/tun2proxy +approaches; likely a targeted POC if boundary preservation is chosen). +Half-answer (hunch): length-prefix each datagram inside the channel +(boundary-preserving), and one channel = one association with per-endpoint +multiplexing inside, mirroring SOCKS5 UDP — but this is exactly the kind +of guess that needs survey + POC before it becomes an ADR. + +### OQ-TN-03: Direction semantics (`-L` / `-R` / dynamic) + +SSH has three forwarding flavors; the crate must model them without +"server/client" framing: + +- `-L` (local forward): consumer dials a local port; producer dials the + target. Channels flows consumer→producer; target dial happens on the + producer side. This is the POC's shape. +- `-R` (remote forward): producer (or a third party) listens; the + *consumer's* side dials or accepts incoming connections and asks the + other side to carry them. Channels flows producer→consumer. +- `-D` (dynamic/SOCKS): one side runs a SOCKS5 server; the target is + chosen per-connection by the client. Addressing arrives per-channel, + not per-tunnel-registration. + +Both sides can be producer and consumer simultaneously (alkcall ADR-022/037 +direction semantics), so the model must not bake direction into the +connection. The open questions: + +- Is direction a field in `params`, or two distinct open-handler shapes / + ALPNs? +- How does `-R` register availability (the side that will carry traffic + advertises listen targets)? Does it interact with `channel/open` at all, + or is it a call-level operation ("please open a tunnel channel to me + when a local accept happens")? +- Dynamic (-D) may not be a tunnel concern at all — it may compose as + "SOCKS5 server implemented over alktunnels dial primitives" in a + separate crate. Keep or cut for v1? + +**Status:** open — needs architecture decision. Half-answer (hunch): `-L` +is the channel/open handler; `-R` needs a small advertisement/lifecycle +surface; `-D` composes on top and is out of scope for the base crate. + +### OQ-TN-04: No forced local binding + +A tunnel must not require the producer (or consumer) to bind a local port. +The POC's shape dialed a target from the handler; binding is optional and +belongs to the caller (assembly layer), not the protocol crate. The API +surface must support: + +- Dial flows with no local bind (POC shape) — covered. +- Listen flows where the binding happens on one side only. +- Unbound/abstract flows (e.g. unix socketpair-style, stdio bridges, + in-process pipes) where neither side binds. + +The protocol layer must express "carry bytes between this target and this +channel" without assuming either endpoint is a bound socket. Substrate +modules (behind feature flags) own actual `bind()` calls; the protocol +owns bookkeeping only. + +**Status:** open — mostly a spec-level requirement to encode in the +architecture docs and API shapes rather than a research question. Half- +answer: already agreed as a requirement (AGENTS.md convention 9); what's +missing is the concrete API surface (who calls what to start a tunnel in +each mode). + +### OQ-TN-05: Backend inversion point — is there a `TunnelBackend` trait? + +alktty has `TtyBackend` because backends (local PTY, docker, SSH) produce +handles and the adapter pumps them. For tunnels, the producer side's +substrate action is narrower — dial a target, or accept on a listener — +so the question: + +- Is a `TunnelBackend`-style trait needed at all, or is the two-pump + handler + feature-gated substrate modules (dial/listen helpers) the + whole story, with the assembly layer wiring substrate streams directly? +- If a trait: what is the handle type? A tunnel "handle" is just an + `AsyncRead + AsyncWrite` stream (or a datagram endpoint) — much thinner + than `TtyHandle`'s stdin/stdout/stderr/exit-code quadruple. The trait + may collapse to "produce a boxed stream for this target" plus a + listener variant. +- Backpressure/limits come from channels (AGENTS.md convention 10); the + backend trait must not add a second layer of them. + +**Status:** open — needs a survey of what backends would actually +implement (local TCP? docker exec? ssh -w?) before deciding trait vs +no-trait. Half-answer (hunch): a thin trait (or just a fn alias) for +"obtain a bidirectional substrate stream for a target," possibly no +trait at all if the only meaningful backends are local sockets — decide +after surveying candidate backends. + +### OQ-TN-06: The two-pump helper — extract now? + +alknet ADR-078 deferred helper extraction until a second two-pump consumer +exists ("a genuine deferral... the contract is decided (shutdown-on- +completion), only the extraction is deferred"). This crate is that second +consumer (POC tunnel was the first; SSH `direct-tcpip` would be a third). + +- Does the helper live here (as a pub utility other handler crates can + use), or upstream in alkcall (which already owns `core` types)? +- Shape: `pump_bidi(recv, send) -> (Future, Future)` returning both + pumps with the shutdown-on-completion wired in? Or a + `join_two_pumps(a, b)` combinator? +- alknet ADR-057 (two-pump helper extraction OQ) noted the helper from + one consumer would bake in a wrong shape; with two consumers the shapes + should be compared before extraction. + +**Status:** open — decide when the first real tunnel handler is written; +not a blocker for the spec. Half-answer: the helper probably belongs +upstream (alkcall, near the channels-adapter handler-integration +conventions) but only if the two shapes genuinely converge. + +### OQ-TN-07: ALPN strategy + +This crate owns the `alk/tunnel`-family ALPN(s). alkcall ADR-004: one +ALPN per protocol; `alk/` prefix. If stream (TCP/unix) and datagram (UDP) +tunnels get distinct ALPNs, the split must be decided before the first +consumer — ALPN strings are wire-stable once published. + +- Option A: single `alk/tunnel` ALPN; substrate is a `params` field + (and datagram framing, if any, is self-describing inside the channel). +- Option B: `alk/tunnel` (stream) + `alk/tunnel-dgram` (datagram), so + the wire framing differs per ALPN cleanly. +- Channels' `params` is ALPN-specific, and the open-handler registry + dispatches per ALPN — both options are cheap mechanically; the cost is + consumer-side API bifurcation (two session types vs one with a + substrate enum). + +**Status:** open — needs the OQ-TN-02 outcome first (if datagrams need +different framing, option B gets stronger). + +### OQ-TN-08: Access control and ownership scope + +Tunnels reach local networks — the open gate is the security boundary. +Shape follows alktty: `TUNNEL_OPEN_SCOPE` scope-gate, and the channels +path gets `AccessControl` wiring for free via +`ChannelCore::register_openable`. Open sub-questions: + +- Should ownership (`OwnershipProvider.owns(...)`) be consulted for + tunnel targets, and what is the resource identity of a tunnel target + (a `host:port`? a registered tunnel name?), given targets may be + arbitrary strings and wildcard targets (`0.0.0.0/0`-style egress) may + be intentionally allowed for some identities? +- Is there a target-allowlist concept (per-identity reachable target + sets), and does it live in `AccessControl` or in the open handler's + params validation? + +**Status:** open — needs alkcall ADR-050 review + a survey of how +alktty scoped its gate. Half-answer (hunch): scope-gate for the open +plus an open-handler-level target policy hook; ownership for +*registered/listened* tunnels (which are persistent resources), not for +ephemeral dials. + +### OQ-TN-09: Lifecycle, teardown, and error reporting + +The two-pump shape gives byte-level teardown for free (EOF sentinels; +channels drops per-channel senders on transport EOF — alknet ADR-078, +POC issue #6). What's missing is the error/level above bytes: + +- How does a failed target dial reach the consumer (e.g. "connection + refused to 10.0.0.5:80")? Is there a structured error frame in the + channel before close, a `channel/close` with reason, or call-level + error on the open op? +- Is there a "tunnel established/failed" ack before byte pumping starts + (alktty has the negotiation frame; the POC's tunnel handler had + nothing — it dialed and pumped)? +- Half-open semantics: one direction EOFs, the other keeps pumping + (standard two-pump behavior) — is that always desired, or does the + consumer need a "close both" control? + +**Status:** open — needs a wire-format decision (ADR) if an error frame +is added. Half-answer (hunch): a self-contained control frame (alktty +ADR-006 shape) carrying an establishment result/error, sent before any +data chunk; dial errors are tunnel-closing (the whole channel dies), +whereas byte-level EOFs stay per-direction. + +### OQ-TN-10: POC scope for what remains unvalidated + +The alknet-channels POC validated TCP only. Candidate targeted POCs +Phase 0 may need (in rough priority order, per the SDD process's +"validate promising approaches"): + +1. **UDP tunnel POC** — boundary-preserving length framing over a + channels channel, per-endpoint multiplexing inside one association, + backpressure behavior. Derisks OQ-TN-02 (and OQ-TN-07's option B). +2. **Reverse-flow POC** — `-R`-style: the accept side listens, the far + side carries. Derisks OQ-TN-03's advertisement/lifecycle shape. +3. **Unix socket + stdio bridge POC** — cheap; validates "substrate + agnostic" beyond IP substrates. +4. **Two-pump helper extraction spike** — OQ-TN-06, only after 1–3. + +POCs live in `.worktrees/research//` per the SDD process, or as +standalone crates (`/workspace/alknet-channels-poc` precedent). + +**Status:** open — pick 1 (and probably 2) after the research pass; +3 is cheap enough to fold into whichever POC runs first. + +## Survey / prior-art list + +Candidate reading for the research specialist (to be expanded): + +- SSH channel/forwarding model: RFC 4254 §7 (direct-tcpip / + forwarded-tcpip), OpenSSH `-L`/`-R`/`-D` semantics, russh's + `ChannelOpen` framing (russh is already in `/workspace/russh`). +- SOCKS5 (RFC 1928): addressing (ATYP), UDP ASSOCIATE framing, + per-endpoint multiplexing — the closest standardized "arbitrary + tunnel + UDP" model. +- tun2proxy (`/workspace/tun2proxy`): UDP-over-TCP tunnel framing in + production; also handles DNS over tunnel. +- quinn-proxy-poc (`/workspace/quinn-proxy-poc`) and iroh + (`/workspace/iroh`): datagram-native transports; how they model + per-endpoint flows. +- alknet docs: ADR-071 §ALPN table (`alknet/tunnel` row), ADR-078, + `docs/architecture/crates/channels/channel-operations.md` (`params` + for `alknet/tunnel` is "the target resource"), and the hub-relay + interaction (ADR-042/079). +- alktty: `NegotiateRequest` shape (self-contained negotiation + precedent), `TtyBackend` inversion point, `TTY_OPEN_SCOPE` access + gate. + +## Convergence checklist (what Phase 0 must produce) + +- [ ] Survey notes: SSH/SOCKS5/tun2proxy addressing + UDP framing + (OQ-TN-01, OQ-TN-02) +- [ ] Recommendation: addressing format sketch (OQ-TN-01) +- [ ] Recommendation: datagram strategy (OQ-TN-02) + ALPN strategy + dependent on it (OQ-TN-07) +- [ ] Recommendation: direction model (-L/-R/-D) (OQ-TN-03) + API + surface sketch satisfying no-forced-binding (OQ-TN-04) +- [ ] Decision input: backend trait vs no-trait (OQ-TN-05) +- [ ] Targeted POC(s) run + summary (OQ-TN-10) — UDP first, reverse + flow second +- [ ] Open questions promoted to Phase 1 + `docs/architecture/open-questions.md` with statuses \ No newline at end of file