Files
alktunnels/docs/research/phase-0-findings.md
T
glm-5.3-flash c2d5cbbbd9 docs: phase 0 research findings — open questions OQ-TN-01..10
Captures the ten open design questions from the setup discussion:
addressing format, UDP datagram semantics, -L/-R/-D direction model,
no-forced-binding API surface, backend inversion point, two-pump
helper extraction, ALPN strategy, access control scope, lifecycle/
error reporting, and candidate targeted POCs. Includes the settled
foundation (POC-validated prior art), a prior-art survey list, and
the Phase 0 convergence checklist.
2026-09-05 19:48:14 +00:00

344 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
status: draft
last_updated: 2026-09-05
---
# alktunnels — Phase 0 Research Findings
This document captures Phase 0 (Exploration) findings and open design
questions for the `alktunnels` crate. The objective of Phase 0 per
`docs/sdd_process.md` is: *"Capture vision and guiding principles; research
options; validate approaches; converge on a recommended approach."* It is the
input to Phase 1 (Architecture), where the Architect will produce
`docs/architecture/` specs, ADRs, and open questions.
Drafted 2026-09-05, emerging from the initial setup discussion. The crate is
the sibling of alktty (`alk/tty` — terminal sessions) on the alkcall
substrate: where alktty multiplexes one service with a fixed five-stream
channel structure, alktunnels generalizes the tunnel handler shape to
arbitrary bidirectional tunnels in the `ssh -L` / `ssh -D` sense — TCP, UDP,
unix sockets, and other stream or datagram substrates.
## What is already settled
The foundation is POC-validated and ADR-pinned; this crate is not starting
from zero. It inherits:
- **The demux→Connection→handler→mux path** — validated by the
alknet-channels POC (Target 3), now production alkcall channels. The
tunnel payload is raw bytes inside a channels data channel; channels
strips its 8-byte header transparently (alknet ADR-093 / alkcall
ADR-035).
- **The two-pump handler shape** — one pump per direction, each pump MUST
shut down the opposite sink on completion (`try_join!` alone deadlocks;
alknet ADR-078). POC-validated with a 1 MiB backpressure test. This
crate is the *second* two-pump consumer the ADR deferred helper
extraction for (the first was the POC's tunnel handler; SSH
`direct-tcpip` would be a later third).
- **The producer/consumer model** — producer registers openable channels
via `ChannelCore::register_openable` (authorization for free via
`AccessControl`); consumer opens tunnel channels via `ChannelClient`
(alkcall ADR-037, ADR-043). Connection direction is independent of
tunnel direction.
- **The backend inversion point pattern** — substrate-specific types
(`TcpStream`, `UdpSocket`, unix sockets) confined to feature-gated
backend modules, injected at the assembly layer, never imported from
the shared/producer/consumer modules (alktty `TtyBackend` precedent).
- **The wasm-clean default crate** — protocol-only code compiles to
`wasm32-unknown-unknown`; socket/platform I/O is feature-gated
(alktty precedent).
- **The relay story** — tunnels traverse alkcall hub relays
transparently via byte-for-byte data-channel forwarding with ID
rewrite (alkcall ADR-042). No tunnel-specific relay work.
## Open Questions
These are the design questions Phase 0 must resolve (or explicitly defer)
before the architecture spec. They are numbered OQ-TN-01.. so they can be
referenced, tracked, and promoted into `docs/architecture/open-questions.md`
in Phase 1. Half-answers and hunches are marked as such — the point of this
document is to hold them without forcing premature decisions.
### OQ-TN-01: Target addressing format
What does the tunnel `params` on `channel/open` look like? alknet ADR-071
§ALPN table noted `alknet/tunnel` as `[0, 1]` data in/out only, but the
addressing scheme was never decided. It must cover at minimum:
- TCP dial (`host:port`)
- UDP (associate-style or endpoint-style — see OQ-TN-02)
- Unix domain sockets (path)
- Direction (who dials the target — see OQ-TN-03)
- Bind/listen vs dial semantics (see OQ-TN-04)
Considerations:
- `params` is ALPN-specific JSON, interpreted by the open handler, not by
the channels layer (alknet ADR-075 / alkcall ADR-039). alktty's
precedent is the `NegotiateRequest` shape — a self-contained JSON
object carried in the open op.
- The addressing string is wire-stable once a consumer exists (one-way
door). It must be substrate-extensible without format changes (a new
substrate should be an additive `scheme` value, not a v2 format).
- Prior art to survey: SSH forwarding models (`direct-tcpip`,
`forwarded-tcpip`, `direct-udpip` in some implementations), SOCKS5
addressing (ATYP + addr + port — supports v4/v6/domain + UDP associate),
iroh/tun2proxy target encoding, quinn-proxy-poc.
**Status:** open — research needed. Half-answer (hunch): a scheme-tagged
JSON object rather than a URL-ish string, so params stay typed and
extensible; exact shape TBD.
### OQ-TN-02: Datagram substrates (UDP) — boundary preservation
Does a UDP tunnel preserve datagram boundaries end-to-end, or does the
tunnel present a byte-stream abstraction to the consumer (boundaries lost,
re-chunked arbitrarily)?
- Channels is a chunk stream with bounded buffers; the zero-length chunk
is the EOF sentinel — datagram boundaries are *not* preserved by the
substrate (alknet ADR-071/093; the POC only exercised TCP).
- SSH's `-D` UDP associate tunnels UDP as a stream with per-datagram
framing re-added by the tunnel protocol (e.g. SOCKS5 UDP over TCP).
russh/openssh do this differently — survey needed.
- iroh and quinn-proxy-poc have native datagram transports; tun2proxy has
a full UDP-over-TCP model worth reading.
- Boundary preservation is a wire-format decision (per-datagram length
framing inside the `BiStream`) and would need an ADR + possibly a BAST
document (AGENTS.md convention 12). Boundary loss is cheaper but
changes what protocols can ride the tunnel (DNS? QUIC? game traffic?).
- Datagrams also raise multiplexing questions TCP does not: one UDP
"association" carries many remote endpoints — does one tunnel channel
carry one endpoint or many, and how are per-endpoint replies routed?
**Status:** open — research needed (survey SSH/russh/SOCKS5/tun2proxy
approaches; likely a targeted POC if boundary preservation is chosen).
Half-answer (hunch): length-prefix each datagram inside the channel
(boundary-preserving), and one channel = one association with per-endpoint
multiplexing inside, mirroring SOCKS5 UDP — but this is exactly the kind
of guess that needs survey + POC before it becomes an ADR.
### OQ-TN-03: Direction semantics (`-L` / `-R` / dynamic)
SSH has three forwarding flavors; the crate must model them without
"server/client" framing:
- `-L` (local forward): consumer dials a local port; producer dials the
target. Channels flows consumer→producer; target dial happens on the
producer side. This is the POC's shape.
- `-R` (remote forward): producer (or a third party) listens; the
*consumer's* side dials or accepts incoming connections and asks the
other side to carry them. Channels flows producer→consumer.
- `-D` (dynamic/SOCKS): one side runs a SOCKS5 server; the target is
chosen per-connection by the client. Addressing arrives per-channel,
not per-tunnel-registration.
Both sides can be producer and consumer simultaneously (alkcall ADR-022/037
direction semantics), so the model must not bake direction into the
connection. The open questions:
- Is direction a field in `params`, or two distinct open-handler shapes /
ALPNs?
- How does `-R` register availability (the side that will carry traffic
advertises listen targets)? Does it interact with `channel/open` at all,
or is it a call-level operation ("please open a tunnel channel to me
when a local accept happens")?
- Dynamic (-D) may not be a tunnel concern at all — it may compose as
"SOCKS5 server implemented over alktunnels dial primitives" in a
separate crate. Keep or cut for v1?
**Status:** open — needs architecture decision. Half-answer (hunch): `-L`
is the channel/open handler; `-R` needs a small advertisement/lifecycle
surface; `-D` composes on top and is out of scope for the base crate.
### OQ-TN-04: No forced local binding
A tunnel must not require the producer (or consumer) to bind a local port.
The POC's shape dialed a target from the handler; binding is optional and
belongs to the caller (assembly layer), not the protocol crate. The API
surface must support:
- Dial flows with no local bind (POC shape) — covered.
- Listen flows where the binding happens on one side only.
- Unbound/abstract flows (e.g. unix socketpair-style, stdio bridges,
in-process pipes) where neither side binds.
The protocol layer must express "carry bytes between this target and this
channel" without assuming either endpoint is a bound socket. Substrate
modules (behind feature flags) own actual `bind()` calls; the protocol
owns bookkeeping only.
**Status:** open — mostly a spec-level requirement to encode in the
architecture docs and API shapes rather than a research question. Half-
answer: already agreed as a requirement (AGENTS.md convention 9); what's
missing is the concrete API surface (who calls what to start a tunnel in
each mode).
### OQ-TN-05: Backend inversion point — is there a `TunnelBackend` trait?
alktty has `TtyBackend` because backends (local PTY, docker, SSH) produce
handles and the adapter pumps them. For tunnels, the producer side's
substrate action is narrower — dial a target, or accept on a listener —
so the question:
- Is a `TunnelBackend`-style trait needed at all, or is the two-pump
handler + feature-gated substrate modules (dial/listen helpers) the
whole story, with the assembly layer wiring substrate streams directly?
- If a trait: what is the handle type? A tunnel "handle" is just an
`AsyncRead + AsyncWrite` stream (or a datagram endpoint) — much thinner
than `TtyHandle`'s stdin/stdout/stderr/exit-code quadruple. The trait
may collapse to "produce a boxed stream for this target" plus a
listener variant.
- Backpressure/limits come from channels (AGENTS.md convention 10); the
backend trait must not add a second layer of them.
**Status:** open — needs a survey of what backends would actually
implement (local TCP? docker exec? ssh -w?) before deciding trait vs
no-trait. Half-answer (hunch): a thin trait (or just a fn alias) for
"obtain a bidirectional substrate stream for a target," possibly no
trait at all if the only meaningful backends are local sockets — decide
after surveying candidate backends.
### OQ-TN-06: The two-pump helper — extract now?
alknet ADR-078 deferred helper extraction until a second two-pump consumer
exists ("a genuine deferral... the contract is decided (shutdown-on-
completion), only the extraction is deferred"). This crate is that second
consumer (POC tunnel was the first; SSH `direct-tcpip` would be a third).
- Does the helper live here (as a pub utility other handler crates can
use), or upstream in alkcall (which already owns `core` types)?
- Shape: `pump_bidi(recv, send) -> (Future, Future)` returning both
pumps with the shutdown-on-completion wired in? Or a
`join_two_pumps(a, b)` combinator?
- alknet ADR-057 (two-pump helper extraction OQ) noted the helper from
one consumer would bake in a wrong shape; with two consumers the shapes
should be compared before extraction.
**Status:** open — decide when the first real tunnel handler is written;
not a blocker for the spec. Half-answer: the helper probably belongs
upstream (alkcall, near the channels-adapter handler-integration
conventions) but only if the two shapes genuinely converge.
### OQ-TN-07: ALPN strategy
This crate owns the `alk/tunnel`-family ALPN(s). alkcall ADR-004: one
ALPN per protocol; `alk/` prefix. If stream (TCP/unix) and datagram (UDP)
tunnels get distinct ALPNs, the split must be decided before the first
consumer — ALPN strings are wire-stable once published.
- Option A: single `alk/tunnel` ALPN; substrate is a `params` field
(and datagram framing, if any, is self-describing inside the channel).
- Option B: `alk/tunnel` (stream) + `alk/tunnel-dgram` (datagram), so
the wire framing differs per ALPN cleanly.
- Channels' `params` is ALPN-specific, and the open-handler registry
dispatches per ALPN — both options are cheap mechanically; the cost is
consumer-side API bifurcation (two session types vs one with a
substrate enum).
**Status:** open — needs the OQ-TN-02 outcome first (if datagrams need
different framing, option B gets stronger).
### OQ-TN-08: Access control and ownership scope
Tunnels reach local networks — the open gate is the security boundary.
Shape follows alktty: `TUNNEL_OPEN_SCOPE` scope-gate, and the channels
path gets `AccessControl` wiring for free via
`ChannelCore::register_openable`. Open sub-questions:
- Should ownership (`OwnershipProvider.owns(...)`) be consulted for
tunnel targets, and what is the resource identity of a tunnel target
(a `host:port`? a registered tunnel name?), given targets may be
arbitrary strings and wildcard targets (`0.0.0.0/0`-style egress) may
be intentionally allowed for some identities?
- Is there a target-allowlist concept (per-identity reachable target
sets), and does it live in `AccessControl` or in the open handler's
params validation?
**Status:** open — needs alkcall ADR-050 review + a survey of how
alktty scoped its gate. Half-answer (hunch): scope-gate for the open
plus an open-handler-level target policy hook; ownership for
*registered/listened* tunnels (which are persistent resources), not for
ephemeral dials.
### OQ-TN-09: Lifecycle, teardown, and error reporting
The two-pump shape gives byte-level teardown for free (EOF sentinels;
channels drops per-channel senders on transport EOF — alknet ADR-078,
POC issue #6). What's missing is the error/level above bytes:
- How does a failed target dial reach the consumer (e.g. "connection
refused to 10.0.0.5:80")? Is there a structured error frame in the
channel before close, a `channel/close` with reason, or call-level
error on the open op?
- Is there a "tunnel established/failed" ack before byte pumping starts
(alktty has the negotiation frame; the POC's tunnel handler had
nothing — it dialed and pumped)?
- Half-open semantics: one direction EOFs, the other keeps pumping
(standard two-pump behavior) — is that always desired, or does the
consumer need a "close both" control?
**Status:** open — needs a wire-format decision (ADR) if an error frame
is added. Half-answer (hunch): a self-contained control frame (alktty
ADR-006 shape) carrying an establishment result/error, sent before any
data chunk; dial errors are tunnel-closing (the whole channel dies),
whereas byte-level EOFs stay per-direction.
### OQ-TN-10: POC scope for what remains unvalidated
The alknet-channels POC validated TCP only. Candidate targeted POCs
Phase 0 may need (in rough priority order, per the SDD process's
"validate promising approaches"):
1. **UDP tunnel POC** — boundary-preserving length framing over a
channels channel, per-endpoint multiplexing inside one association,
backpressure behavior. Derisks OQ-TN-02 (and OQ-TN-07's option B).
2. **Reverse-flow POC**`-R`-style: the accept side listens, the far
side carries. Derisks OQ-TN-03's advertisement/lifecycle shape.
3. **Unix socket + stdio bridge POC** — cheap; validates "substrate
agnostic" beyond IP substrates.
4. **Two-pump helper extraction spike** — OQ-TN-06, only after 13.
POCs live in `.worktrees/research/<task-id>/` per the SDD process, or as
standalone crates (`/workspace/alknet-channels-poc` precedent).
**Status:** open — pick 1 (and probably 2) after the research pass;
3 is cheap enough to fold into whichever POC runs first.
## Survey / prior-art list
Candidate reading for the research specialist (to be expanded):
- SSH channel/forwarding model: RFC 4254 §7 (direct-tcpip /
forwarded-tcpip), OpenSSH `-L`/`-R`/`-D` semantics, russh's
`ChannelOpen` framing (russh is already in `/workspace/russh`).
- SOCKS5 (RFC 1928): addressing (ATYP), UDP ASSOCIATE framing,
per-endpoint multiplexing — the closest standardized "arbitrary
tunnel + UDP" model.
- tun2proxy (`/workspace/tun2proxy`): UDP-over-TCP tunnel framing in
production; also handles DNS over tunnel.
- quinn-proxy-poc (`/workspace/quinn-proxy-poc`) and iroh
(`/workspace/iroh`): datagram-native transports; how they model
per-endpoint flows.
- alknet docs: ADR-071 §ALPN table (`alknet/tunnel` row), ADR-078,
`docs/architecture/crates/channels/channel-operations.md` (`params`
for `alknet/tunnel` is "the target resource"), and the hub-relay
interaction (ADR-042/079).
- alktty: `NegotiateRequest` shape (self-contained negotiation
precedent), `TtyBackend` inversion point, `TTY_OPEN_SCOPE` access
gate.
## Convergence checklist (what Phase 0 must produce)
- [ ] Survey notes: SSH/SOCKS5/tun2proxy addressing + UDP framing
(OQ-TN-01, OQ-TN-02)
- [ ] Recommendation: addressing format sketch (OQ-TN-01)
- [ ] Recommendation: datagram strategy (OQ-TN-02) + ALPN strategy
dependent on it (OQ-TN-07)
- [ ] Recommendation: direction model (-L/-R/-D) (OQ-TN-03) + API
surface sketch satisfying no-forced-binding (OQ-TN-04)
- [ ] Decision input: backend trait vs no-trait (OQ-TN-05)
- [ ] Targeted POC(s) run + summary (OQ-TN-10) — UDP first, reverse
flow second
- [ ] Open questions promoted to Phase 1
`docs/architecture/open-questions.md` with statuses