docs: phase 0 research findings — open questions OQ-TN-01..10

Captures the ten open design questions from the setup discussion: addressing format, UDP datagram semantics, -L/-R/-D direction model, no-forced-binding API surface, backend inversion point, two-pump helper extraction, ALPN strategy, access control scope, lifecycle/ error reporting, and candidate targeted POCs. Includes the settled foundation (POC-validated prior art), a prior-art survey list, and the Phase 0 convergence checklist.
2026-09-05 19:48:14 +00:00
parent 43a5e4204e
commit c2d5cbbbd9
1 changed files with 344 additions and 0 deletions
@@ -0,0 +1,344 @@
 ---
 status: draft
 last_updated: 2026-09-05
 ---
 # alktunnels — Phase 0 Research Findings
 This document captures Phase 0 (Exploration) findings and open design
 questions for the `alktunnels` crate. The objective of Phase 0 per
 `docs/sdd_process.md` is: *"Capture vision and guiding principles; research
 options; validate approaches; converge on a recommended approach."* It is the
 input to Phase 1 (Architecture), where the Architect will produce
 `docs/architecture/` specs, ADRs, and open questions.
 Drafted 2026-09-05, emerging from the initial setup discussion. The crate is
 the sibling of alktty (`alk/tty` — terminal sessions) on the alkcall
 substrate: where alktty multiplexes one service with a fixed five-stream
 channel structure, alktunnels generalizes the tunnel handler shape to
 arbitrary bidirectional tunnels in the `ssh -L` / `ssh -D` sense — TCP, UDP,
 unix sockets, and other stream or datagram substrates.
 ## What is already settled
 The foundation is POC-validated and ADR-pinned; this crate is not starting
 from zero. It inherits:
 - **The demux→Connection→handler→mux path** — validated by the
  alknet-channels POC (Target 3), now production alkcall channels. The
  tunnel payload is raw bytes inside a channels data channel; channels
  strips its 8-byte header transparently (alknet ADR-093 / alkcall
  ADR-035).
 - **The two-pump handler shape** — one pump per direction, each pump MUST
  shut down the opposite sink on completion (`try_join!` alone deadlocks;
  alknet ADR-078). POC-validated with a 1 MiB backpressure test. This
  crate is the *second* two-pump consumer the ADR deferred helper
  extraction for (the first was the POC's tunnel handler; SSH
  `direct-tcpip` would be a later third).
 - **The producer/consumer model** — producer registers openable channels
  via `ChannelCore::register_openable` (authorization for free via
  `AccessControl`); consumer opens tunnel channels via `ChannelClient`
  (alkcall ADR-037, ADR-043). Connection direction is independent of
  tunnel direction.
 - **The backend inversion point pattern** — substrate-specific types
  (`TcpStream`, `UdpSocket`, unix sockets) confined to feature-gated
  backend modules, injected at the assembly layer, never imported from
  the shared/producer/consumer modules (alktty `TtyBackend` precedent).
 - **The wasm-clean default crate** — protocol-only code compiles to
  `wasm32-unknown-unknown`; socket/platform I/O is feature-gated
  (alktty precedent).
 - **The relay story** — tunnels traverse alkcall hub relays
  transparently via byte-for-byte data-channel forwarding with ID
  rewrite (alkcall ADR-042). No tunnel-specific relay work.
 ## Open Questions
 These are the design questions Phase 0 must resolve (or explicitly defer)
 before the architecture spec. They are numbered OQ-TN-01.. so they can be
 referenced, tracked, and promoted into `docs/architecture/open-questions.md`
 in Phase 1. Half-answers and hunches are marked as such — the point of this
 document is to hold them without forcing premature decisions.
 ### OQ-TN-01: Target addressing format
 What does the tunnel `params` on `channel/open` look like? alknet ADR-071
 §ALPN table noted `alknet/tunnel` as `[0, 1]` data in/out only, but the
 addressing scheme was never decided. It must cover at minimum:
 - TCP dial (`host:port`)
 - UDP (associate-style or endpoint-style — see OQ-TN-02)
 - Unix domain sockets (path)
 - Direction (who dials the target — see OQ-TN-03)
 - Bind/listen vs dial semantics (see OQ-TN-04)
 Considerations:
 - `params` is ALPN-specific JSON, interpreted by the open handler, not by
  the channels layer (alknet ADR-075 / alkcall ADR-039). alktty's
  precedent is the `NegotiateRequest` shape — a self-contained JSON
  object carried in the open op.
 - The addressing string is wire-stable once a consumer exists (one-way
  door). It must be substrate-extensible without format changes (a new
  substrate should be an additive `scheme` value, not a v2 format).
 - Prior art to survey: SSH forwarding models (`direct-tcpip`,
  `forwarded-tcpip`, `direct-udpip` in some implementations), SOCKS5
  addressing (ATYP + addr + port — supports v4/v6/domain + UDP associate),
  iroh/tun2proxy target encoding, quinn-proxy-poc.
 **Status:** open — research needed. Half-answer (hunch): a scheme-tagged
 JSON object rather than a URL-ish string, so params stay typed and
 extensible; exact shape TBD.
 ### OQ-TN-02: Datagram substrates (UDP) — boundary preservation
 Does a UDP tunnel preserve datagram boundaries end-to-end, or does the
 tunnel present a byte-stream abstraction to the consumer (boundaries lost,
 re-chunked arbitrarily)?
 - Channels is a chunk stream with bounded buffers; the zero-length chunk
  is the EOF sentinel — datagram boundaries are *not* preserved by the
  substrate (alknet ADR-071/093; the POC only exercised TCP).
 - SSH's `-D` UDP associate tunnels UDP as a stream with per-datagram
  framing re-added by the tunnel protocol (e.g. SOCKS5 UDP over TCP).
  russh/openssh do this differently — survey needed.
 - iroh and quinn-proxy-poc have native datagram transports; tun2proxy has
  a full UDP-over-TCP model worth reading.
 - Boundary preservation is a wire-format decision (per-datagram length
  framing inside the `BiStream`) and would need an ADR + possibly a BAST
  document (AGENTS.md convention 12). Boundary loss is cheaper but
  changes what protocols can ride the tunnel (DNS? QUIC? game traffic?).
 - Datagrams also raise multiplexing questions TCP does not: one UDP
  "association" carries many remote endpoints — does one tunnel channel
  carry one endpoint or many, and how are per-endpoint replies routed?
 **Status:** open — research needed (survey SSH/russh/SOCKS5/tun2proxy
 approaches; likely a targeted POC if boundary preservation is chosen).
 Half-answer (hunch): length-prefix each datagram inside the channel
 (boundary-preserving), and one channel = one association with per-endpoint
 multiplexing inside, mirroring SOCKS5 UDP — but this is exactly the kind
 of guess that needs survey + POC before it becomes an ADR.
 ### OQ-TN-03: Direction semantics (`-L` / `-R` / dynamic)
 SSH has three forwarding flavors; the crate must model them without
 "server/client" framing:
 - `-L` (local forward): consumer dials a local port; producer dials the
  target. Channels flows consumer→producer; target dial happens on the
  producer side. This is the POC's shape.
 - `-R` (remote forward): producer (or a third party) listens; the
  *consumer's* side dials or accepts incoming connections and asks the
  other side to carry them. Channels flows producer→consumer.
 - `-D` (dynamic/SOCKS): one side runs a SOCKS5 server; the target is
  chosen per-connection by the client. Addressing arrives per-channel,
  not per-tunnel-registration.
 Both sides can be producer and consumer simultaneously (alkcall ADR-022/037
 direction semantics), so the model must not bake direction into the
 connection. The open questions:
 - Is direction a field in `params`, or two distinct open-handler shapes /
  ALPNs?
 - How does `-R` register availability (the side that will carry traffic
  advertises listen targets)? Does it interact with `channel/open` at all,
  or is it a call-level operation ("please open a tunnel channel to me
  when a local accept happens")?
 - Dynamic (-D) may not be a tunnel concern at all — it may compose as
  "SOCKS5 server implemented over alktunnels dial primitives" in a
  separate crate. Keep or cut for v1?
 **Status:** open — needs architecture decision. Half-answer (hunch): `-L`
 is the channel/open handler; `-R` needs a small advertisement/lifecycle
 surface; `-D` composes on top and is out of scope for the base crate.
 ### OQ-TN-04: No forced local binding
 A tunnel must not require the producer (or consumer) to bind a local port.
 The POC's shape dialed a target from the handler; binding is optional and
 belongs to the caller (assembly layer), not the protocol crate. The API
 surface must support:
 - Dial flows with no local bind (POC shape) — covered.
 - Listen flows where the binding happens on one side only.
 - Unbound/abstract flows (e.g. unix socketpair-style, stdio bridges,
  in-process pipes) where neither side binds.
 The protocol layer must express "carry bytes between this target and this
 channel" without assuming either endpoint is a bound socket. Substrate
 modules (behind feature flags) own actual `bind()` calls; the protocol
 owns bookkeeping only.
 **Status:** open — mostly a spec-level requirement to encode in the
 architecture docs and API shapes rather than a research question. Half-
 answer: already agreed as a requirement (AGENTS.md convention 9); what's
 missing is the concrete API surface (who calls what to start a tunnel in
 each mode).
 ### OQ-TN-05: Backend inversion point — is there a `TunnelBackend` trait?
 alktty has `TtyBackend` because backends (local PTY, docker, SSH) produce
 handles and the adapter pumps them. For tunnels, the producer side's
 substrate action is narrower — dial a target, or accept on a listener —
 so the question:
 - Is a `TunnelBackend`-style trait needed at all, or is the two-pump
  handler + feature-gated substrate modules (dial/listen helpers) the
  whole story, with the assembly layer wiring substrate streams directly?
 - If a trait: what is the handle type? A tunnel "handle" is just an
  `AsyncRead + AsyncWrite` stream (or a datagram endpoint) — much thinner
  than `TtyHandle`'s stdin/stdout/stderr/exit-code quadruple. The trait
  may collapse to "produce a boxed stream for this target" plus a
  listener variant.
 - Backpressure/limits come from channels (AGENTS.md convention 10); the
  backend trait must not add a second layer of them.
 **Status:** open — needs a survey of what backends would actually
 implement (local TCP? docker exec? ssh -w?) before deciding trait vs
 no-trait. Half-answer (hunch): a thin trait (or just a fn alias) for
 "obtain a bidirectional substrate stream for a target," possibly no
 trait at all if the only meaningful backends are local sockets — decide
 after surveying candidate backends.
 ### OQ-TN-06: The two-pump helper — extract now?
 alknet ADR-078 deferred helper extraction until a second two-pump consumer
 exists ("a genuine deferral... the contract is decided (shutdown-on-
 completion), only the extraction is deferred"). This crate is that second
 consumer (POC tunnel was the first; SSH `direct-tcpip` would be a third).
 - Does the helper live here (as a pub utility other handler crates can
  use), or upstream in alkcall (which already owns `core` types)?
 - Shape: `pump_bidi(recv, send) -> (Future, Future)` returning both
  pumps with the shutdown-on-completion wired in? Or a
  `join_two_pumps(a, b)` combinator?
 - alknet ADR-057 (two-pump helper extraction OQ) noted the helper from
  one consumer would bake in a wrong shape; with two consumers the shapes
  should be compared before extraction.
 **Status:** open — decide when the first real tunnel handler is written;
 not a blocker for the spec. Half-answer: the helper probably belongs
 upstream (alkcall, near the channels-adapter handler-integration
 conventions) but only if the two shapes genuinely converge.
 ### OQ-TN-07: ALPN strategy
 This crate owns the `alk/tunnel`-family ALPN(s). alkcall ADR-004: one
 ALPN per protocol; `alk/` prefix. If stream (TCP/unix) and datagram (UDP)
 tunnels get distinct ALPNs, the split must be decided before the first
 consumer — ALPN strings are wire-stable once published.
 - Option A: single `alk/tunnel` ALPN; substrate is a `params` field
  (and datagram framing, if any, is self-describing inside the channel).
 - Option B: `alk/tunnel` (stream) + `alk/tunnel-dgram` (datagram), so
  the wire framing differs per ALPN cleanly.
 - Channels' `params` is ALPN-specific, and the open-handler registry
  dispatches per ALPN — both options are cheap mechanically; the cost is
  consumer-side API bifurcation (two session types vs one with a
  substrate enum).
 **Status:** open — needs the OQ-TN-02 outcome first (if datagrams need
 different framing, option B gets stronger).
 ### OQ-TN-08: Access control and ownership scope
 Tunnels reach local networks — the open gate is the security boundary.
 Shape follows alktty: `TUNNEL_OPEN_SCOPE` scope-gate, and the channels
 path gets `AccessControl` wiring for free via
 `ChannelCore::register_openable`. Open sub-questions:
 - Should ownership (`OwnershipProvider.owns(...)`) be consulted for
  tunnel targets, and what is the resource identity of a tunnel target
  (a `host:port`? a registered tunnel name?), given targets may be
  arbitrary strings and wildcard targets (`0.0.0.0/0`-style egress) may
  be intentionally allowed for some identities?
 - Is there a target-allowlist concept (per-identity reachable target
  sets), and does it live in `AccessControl` or in the open handler's
  params validation?
 **Status:** open — needs alkcall ADR-050 review + a survey of how
 alktty scoped its gate. Half-answer (hunch): scope-gate for the open
 plus an open-handler-level target policy hook; ownership for
 *registered/listened* tunnels (which are persistent resources), not for
 ephemeral dials.
 ### OQ-TN-09: Lifecycle, teardown, and error reporting
 The two-pump shape gives byte-level teardown for free (EOF sentinels;
 channels drops per-channel senders on transport EOF — alknet ADR-078,
 POC issue #6). What's missing is the error/level above bytes:
 - How does a failed target dial reach the consumer (e.g. "connection
  refused to 10.0.0.5:80")? Is there a structured error frame in the
  channel before close, a `channel/close` with reason, or call-level
  error on the open op?
 - Is there a "tunnel established/failed" ack before byte pumping starts
  (alktty has the negotiation frame; the POC's tunnel handler had
  nothing — it dialed and pumped)?
 - Half-open semantics: one direction EOFs, the other keeps pumping
  (standard two-pump behavior) — is that always desired, or does the
  consumer need a "close both" control?
 **Status:** open — needs a wire-format decision (ADR) if an error frame
 is added. Half-answer (hunch): a self-contained control frame (alktty
 ADR-006 shape) carrying an establishment result/error, sent before any
 data chunk; dial errors are tunnel-closing (the whole channel dies),
 whereas byte-level EOFs stay per-direction.
 ### OQ-TN-10: POC scope for what remains unvalidated
 The alknet-channels POC validated TCP only. Candidate targeted POCs
 Phase 0 may need (in rough priority order, per the SDD process's
 "validate promising approaches"):
 1. **UDP tunnel POC** — boundary-preserving length framing over a
   channels channel, per-endpoint multiplexing inside one association,
   backpressure behavior. Derisks OQ-TN-02 (and OQ-TN-07's option B).
 2. **Reverse-flow POC** — `-R`-style: the accept side listens, the far
   side carries. Derisks OQ-TN-03's advertisement/lifecycle shape.
 3. **Unix socket + stdio bridge POC** — cheap; validates "substrate
   agnostic" beyond IP substrates.
 4. **Two-pump helper extraction spike** — OQ-TN-06, only after 1–3.
 POCs live in `.worktrees/research/<task-id>/` per the SDD process, or as
 standalone crates (`/workspace/alknet-channels-poc` precedent).
 **Status:** open — pick 1 (and probably 2) after the research pass;
 3 is cheap enough to fold into whichever POC runs first.
 ## Survey / prior-art list
 Candidate reading for the research specialist (to be expanded):
 - SSH channel/forwarding model: RFC 4254 §7 (direct-tcpip /
  forwarded-tcpip), OpenSSH `-L`/`-R`/`-D` semantics, russh's
  `ChannelOpen` framing (russh is already in `/workspace/russh`).
 - SOCKS5 (RFC 1928): addressing (ATYP), UDP ASSOCIATE framing,
  per-endpoint multiplexing — the closest standardized "arbitrary
  tunnel + UDP" model.
 - tun2proxy (`/workspace/tun2proxy`): UDP-over-TCP tunnel framing in
  production; also handles DNS over tunnel.
 - quinn-proxy-poc (`/workspace/quinn-proxy-poc`) and iroh
  (`/workspace/iroh`): datagram-native transports; how they model
  per-endpoint flows.
 - alknet docs: ADR-071 §ALPN table (`alknet/tunnel` row), ADR-078,
  `docs/architecture/crates/channels/channel-operations.md` (`params`
  for `alknet/tunnel` is "the target resource"), and the hub-relay
  interaction (ADR-042/079).
 - alktty: `NegotiateRequest` shape (self-contained negotiation
  precedent), `TtyBackend` inversion point, `TTY_OPEN_SCOPE` access
  gate.
 ## Convergence checklist (what Phase 0 must produce)
 - [ ] Survey notes: SSH/SOCKS5/tun2proxy addressing + UDP framing
      (OQ-TN-01, OQ-TN-02)
 - [ ] Recommendation: addressing format sketch (OQ-TN-01)
 - [ ] Recommendation: datagram strategy (OQ-TN-02) + ALPN strategy
      dependent on it (OQ-TN-07)
 - [ ] Recommendation: direction model (-L/-R/-D) (OQ-TN-03) + API
      surface sketch satisfying no-forced-binding (OQ-TN-04)
 - [ ] Decision input: backend trait vs no-trait (OQ-TN-05)
 - [ ] Targeted POC(s) run + summary (OQ-TN-10) — UDP first, reverse
      flow second
 - [ ] Open questions promoted to Phase 1
      `docs/architecture/open-questions.md` with statuses