docs: phase 0 research findings — open questions OQ-TN-01..10
Captures the ten open design questions from the setup discussion: addressing format, UDP datagram semantics, -L/-R/-D direction model, no-forced-binding API surface, backend inversion point, two-pump helper extraction, ALPN strategy, access control scope, lifecycle/ error reporting, and candidate targeted POCs. Includes the settled foundation (POC-validated prior art), a prior-art survey list, and the Phase 0 convergence checklist.
This commit is contained in:
@@ -0,0 +1,344 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-09-05
|
||||
---
|
||||
|
||||
# alktunnels — Phase 0 Research Findings
|
||||
|
||||
This document captures Phase 0 (Exploration) findings and open design
|
||||
questions for the `alktunnels` crate. The objective of Phase 0 per
|
||||
`docs/sdd_process.md` is: *"Capture vision and guiding principles; research
|
||||
options; validate approaches; converge on a recommended approach."* It is the
|
||||
input to Phase 1 (Architecture), where the Architect will produce
|
||||
`docs/architecture/` specs, ADRs, and open questions.
|
||||
|
||||
Drafted 2026-09-05, emerging from the initial setup discussion. The crate is
|
||||
the sibling of alktty (`alk/tty` — terminal sessions) on the alkcall
|
||||
substrate: where alktty multiplexes one service with a fixed five-stream
|
||||
channel structure, alktunnels generalizes the tunnel handler shape to
|
||||
arbitrary bidirectional tunnels in the `ssh -L` / `ssh -D` sense — TCP, UDP,
|
||||
unix sockets, and other stream or datagram substrates.
|
||||
|
||||
## What is already settled
|
||||
|
||||
The foundation is POC-validated and ADR-pinned; this crate is not starting
|
||||
from zero. It inherits:
|
||||
|
||||
- **The demux→Connection→handler→mux path** — validated by the
|
||||
alknet-channels POC (Target 3), now production alkcall channels. The
|
||||
tunnel payload is raw bytes inside a channels data channel; channels
|
||||
strips its 8-byte header transparently (alknet ADR-093 / alkcall
|
||||
ADR-035).
|
||||
- **The two-pump handler shape** — one pump per direction, each pump MUST
|
||||
shut down the opposite sink on completion (`try_join!` alone deadlocks;
|
||||
alknet ADR-078). POC-validated with a 1 MiB backpressure test. This
|
||||
crate is the *second* two-pump consumer the ADR deferred helper
|
||||
extraction for (the first was the POC's tunnel handler; SSH
|
||||
`direct-tcpip` would be a later third).
|
||||
- **The producer/consumer model** — producer registers openable channels
|
||||
via `ChannelCore::register_openable` (authorization for free via
|
||||
`AccessControl`); consumer opens tunnel channels via `ChannelClient`
|
||||
(alkcall ADR-037, ADR-043). Connection direction is independent of
|
||||
tunnel direction.
|
||||
- **The backend inversion point pattern** — substrate-specific types
|
||||
(`TcpStream`, `UdpSocket`, unix sockets) confined to feature-gated
|
||||
backend modules, injected at the assembly layer, never imported from
|
||||
the shared/producer/consumer modules (alktty `TtyBackend` precedent).
|
||||
- **The wasm-clean default crate** — protocol-only code compiles to
|
||||
`wasm32-unknown-unknown`; socket/platform I/O is feature-gated
|
||||
(alktty precedent).
|
||||
- **The relay story** — tunnels traverse alkcall hub relays
|
||||
transparently via byte-for-byte data-channel forwarding with ID
|
||||
rewrite (alkcall ADR-042). No tunnel-specific relay work.
|
||||
|
||||
## Open Questions
|
||||
|
||||
These are the design questions Phase 0 must resolve (or explicitly defer)
|
||||
before the architecture spec. They are numbered OQ-TN-01.. so they can be
|
||||
referenced, tracked, and promoted into `docs/architecture/open-questions.md`
|
||||
in Phase 1. Half-answers and hunches are marked as such — the point of this
|
||||
document is to hold them without forcing premature decisions.
|
||||
|
||||
### OQ-TN-01: Target addressing format
|
||||
|
||||
What does the tunnel `params` on `channel/open` look like? alknet ADR-071
|
||||
§ALPN table noted `alknet/tunnel` as `[0, 1]` data in/out only, but the
|
||||
addressing scheme was never decided. It must cover at minimum:
|
||||
|
||||
- TCP dial (`host:port`)
|
||||
- UDP (associate-style or endpoint-style — see OQ-TN-02)
|
||||
- Unix domain sockets (path)
|
||||
- Direction (who dials the target — see OQ-TN-03)
|
||||
- Bind/listen vs dial semantics (see OQ-TN-04)
|
||||
|
||||
Considerations:
|
||||
|
||||
- `params` is ALPN-specific JSON, interpreted by the open handler, not by
|
||||
the channels layer (alknet ADR-075 / alkcall ADR-039). alktty's
|
||||
precedent is the `NegotiateRequest` shape — a self-contained JSON
|
||||
object carried in the open op.
|
||||
- The addressing string is wire-stable once a consumer exists (one-way
|
||||
door). It must be substrate-extensible without format changes (a new
|
||||
substrate should be an additive `scheme` value, not a v2 format).
|
||||
- Prior art to survey: SSH forwarding models (`direct-tcpip`,
|
||||
`forwarded-tcpip`, `direct-udpip` in some implementations), SOCKS5
|
||||
addressing (ATYP + addr + port — supports v4/v6/domain + UDP associate),
|
||||
iroh/tun2proxy target encoding, quinn-proxy-poc.
|
||||
|
||||
**Status:** open — research needed. Half-answer (hunch): a scheme-tagged
|
||||
JSON object rather than a URL-ish string, so params stay typed and
|
||||
extensible; exact shape TBD.
|
||||
|
||||
### OQ-TN-02: Datagram substrates (UDP) — boundary preservation
|
||||
|
||||
Does a UDP tunnel preserve datagram boundaries end-to-end, or does the
|
||||
tunnel present a byte-stream abstraction to the consumer (boundaries lost,
|
||||
re-chunked arbitrarily)?
|
||||
|
||||
- Channels is a chunk stream with bounded buffers; the zero-length chunk
|
||||
is the EOF sentinel — datagram boundaries are *not* preserved by the
|
||||
substrate (alknet ADR-071/093; the POC only exercised TCP).
|
||||
- SSH's `-D` UDP associate tunnels UDP as a stream with per-datagram
|
||||
framing re-added by the tunnel protocol (e.g. SOCKS5 UDP over TCP).
|
||||
russh/openssh do this differently — survey needed.
|
||||
- iroh and quinn-proxy-poc have native datagram transports; tun2proxy has
|
||||
a full UDP-over-TCP model worth reading.
|
||||
- Boundary preservation is a wire-format decision (per-datagram length
|
||||
framing inside the `BiStream`) and would need an ADR + possibly a BAST
|
||||
document (AGENTS.md convention 12). Boundary loss is cheaper but
|
||||
changes what protocols can ride the tunnel (DNS? QUIC? game traffic?).
|
||||
- Datagrams also raise multiplexing questions TCP does not: one UDP
|
||||
"association" carries many remote endpoints — does one tunnel channel
|
||||
carry one endpoint or many, and how are per-endpoint replies routed?
|
||||
|
||||
**Status:** open — research needed (survey SSH/russh/SOCKS5/tun2proxy
|
||||
approaches; likely a targeted POC if boundary preservation is chosen).
|
||||
Half-answer (hunch): length-prefix each datagram inside the channel
|
||||
(boundary-preserving), and one channel = one association with per-endpoint
|
||||
multiplexing inside, mirroring SOCKS5 UDP — but this is exactly the kind
|
||||
of guess that needs survey + POC before it becomes an ADR.
|
||||
|
||||
### OQ-TN-03: Direction semantics (`-L` / `-R` / dynamic)
|
||||
|
||||
SSH has three forwarding flavors; the crate must model them without
|
||||
"server/client" framing:
|
||||
|
||||
- `-L` (local forward): consumer dials a local port; producer dials the
|
||||
target. Channels flows consumer→producer; target dial happens on the
|
||||
producer side. This is the POC's shape.
|
||||
- `-R` (remote forward): producer (or a third party) listens; the
|
||||
*consumer's* side dials or accepts incoming connections and asks the
|
||||
other side to carry them. Channels flows producer→consumer.
|
||||
- `-D` (dynamic/SOCKS): one side runs a SOCKS5 server; the target is
|
||||
chosen per-connection by the client. Addressing arrives per-channel,
|
||||
not per-tunnel-registration.
|
||||
|
||||
Both sides can be producer and consumer simultaneously (alkcall ADR-022/037
|
||||
direction semantics), so the model must not bake direction into the
|
||||
connection. The open questions:
|
||||
|
||||
- Is direction a field in `params`, or two distinct open-handler shapes /
|
||||
ALPNs?
|
||||
- How does `-R` register availability (the side that will carry traffic
|
||||
advertises listen targets)? Does it interact with `channel/open` at all,
|
||||
or is it a call-level operation ("please open a tunnel channel to me
|
||||
when a local accept happens")?
|
||||
- Dynamic (-D) may not be a tunnel concern at all — it may compose as
|
||||
"SOCKS5 server implemented over alktunnels dial primitives" in a
|
||||
separate crate. Keep or cut for v1?
|
||||
|
||||
**Status:** open — needs architecture decision. Half-answer (hunch): `-L`
|
||||
is the channel/open handler; `-R` needs a small advertisement/lifecycle
|
||||
surface; `-D` composes on top and is out of scope for the base crate.
|
||||
|
||||
### OQ-TN-04: No forced local binding
|
||||
|
||||
A tunnel must not require the producer (or consumer) to bind a local port.
|
||||
The POC's shape dialed a target from the handler; binding is optional and
|
||||
belongs to the caller (assembly layer), not the protocol crate. The API
|
||||
surface must support:
|
||||
|
||||
- Dial flows with no local bind (POC shape) — covered.
|
||||
- Listen flows where the binding happens on one side only.
|
||||
- Unbound/abstract flows (e.g. unix socketpair-style, stdio bridges,
|
||||
in-process pipes) where neither side binds.
|
||||
|
||||
The protocol layer must express "carry bytes between this target and this
|
||||
channel" without assuming either endpoint is a bound socket. Substrate
|
||||
modules (behind feature flags) own actual `bind()` calls; the protocol
|
||||
owns bookkeeping only.
|
||||
|
||||
**Status:** open — mostly a spec-level requirement to encode in the
|
||||
architecture docs and API shapes rather than a research question. Half-
|
||||
answer: already agreed as a requirement (AGENTS.md convention 9); what's
|
||||
missing is the concrete API surface (who calls what to start a tunnel in
|
||||
each mode).
|
||||
|
||||
### OQ-TN-05: Backend inversion point — is there a `TunnelBackend` trait?
|
||||
|
||||
alktty has `TtyBackend` because backends (local PTY, docker, SSH) produce
|
||||
handles and the adapter pumps them. For tunnels, the producer side's
|
||||
substrate action is narrower — dial a target, or accept on a listener —
|
||||
so the question:
|
||||
|
||||
- Is a `TunnelBackend`-style trait needed at all, or is the two-pump
|
||||
handler + feature-gated substrate modules (dial/listen helpers) the
|
||||
whole story, with the assembly layer wiring substrate streams directly?
|
||||
- If a trait: what is the handle type? A tunnel "handle" is just an
|
||||
`AsyncRead + AsyncWrite` stream (or a datagram endpoint) — much thinner
|
||||
than `TtyHandle`'s stdin/stdout/stderr/exit-code quadruple. The trait
|
||||
may collapse to "produce a boxed stream for this target" plus a
|
||||
listener variant.
|
||||
- Backpressure/limits come from channels (AGENTS.md convention 10); the
|
||||
backend trait must not add a second layer of them.
|
||||
|
||||
**Status:** open — needs a survey of what backends would actually
|
||||
implement (local TCP? docker exec? ssh -w?) before deciding trait vs
|
||||
no-trait. Half-answer (hunch): a thin trait (or just a fn alias) for
|
||||
"obtain a bidirectional substrate stream for a target," possibly no
|
||||
trait at all if the only meaningful backends are local sockets — decide
|
||||
after surveying candidate backends.
|
||||
|
||||
### OQ-TN-06: The two-pump helper — extract now?
|
||||
|
||||
alknet ADR-078 deferred helper extraction until a second two-pump consumer
|
||||
exists ("a genuine deferral... the contract is decided (shutdown-on-
|
||||
completion), only the extraction is deferred"). This crate is that second
|
||||
consumer (POC tunnel was the first; SSH `direct-tcpip` would be a third).
|
||||
|
||||
- Does the helper live here (as a pub utility other handler crates can
|
||||
use), or upstream in alkcall (which already owns `core` types)?
|
||||
- Shape: `pump_bidi(recv, send) -> (Future, Future)` returning both
|
||||
pumps with the shutdown-on-completion wired in? Or a
|
||||
`join_two_pumps(a, b)` combinator?
|
||||
- alknet ADR-057 (two-pump helper extraction OQ) noted the helper from
|
||||
one consumer would bake in a wrong shape; with two consumers the shapes
|
||||
should be compared before extraction.
|
||||
|
||||
**Status:** open — decide when the first real tunnel handler is written;
|
||||
not a blocker for the spec. Half-answer: the helper probably belongs
|
||||
upstream (alkcall, near the channels-adapter handler-integration
|
||||
conventions) but only if the two shapes genuinely converge.
|
||||
|
||||
### OQ-TN-07: ALPN strategy
|
||||
|
||||
This crate owns the `alk/tunnel`-family ALPN(s). alkcall ADR-004: one
|
||||
ALPN per protocol; `alk/` prefix. If stream (TCP/unix) and datagram (UDP)
|
||||
tunnels get distinct ALPNs, the split must be decided before the first
|
||||
consumer — ALPN strings are wire-stable once published.
|
||||
|
||||
- Option A: single `alk/tunnel` ALPN; substrate is a `params` field
|
||||
(and datagram framing, if any, is self-describing inside the channel).
|
||||
- Option B: `alk/tunnel` (stream) + `alk/tunnel-dgram` (datagram), so
|
||||
the wire framing differs per ALPN cleanly.
|
||||
- Channels' `params` is ALPN-specific, and the open-handler registry
|
||||
dispatches per ALPN — both options are cheap mechanically; the cost is
|
||||
consumer-side API bifurcation (two session types vs one with a
|
||||
substrate enum).
|
||||
|
||||
**Status:** open — needs the OQ-TN-02 outcome first (if datagrams need
|
||||
different framing, option B gets stronger).
|
||||
|
||||
### OQ-TN-08: Access control and ownership scope
|
||||
|
||||
Tunnels reach local networks — the open gate is the security boundary.
|
||||
Shape follows alktty: `TUNNEL_OPEN_SCOPE` scope-gate, and the channels
|
||||
path gets `AccessControl` wiring for free via
|
||||
`ChannelCore::register_openable`. Open sub-questions:
|
||||
|
||||
- Should ownership (`OwnershipProvider.owns(...)`) be consulted for
|
||||
tunnel targets, and what is the resource identity of a tunnel target
|
||||
(a `host:port`? a registered tunnel name?), given targets may be
|
||||
arbitrary strings and wildcard targets (`0.0.0.0/0`-style egress) may
|
||||
be intentionally allowed for some identities?
|
||||
- Is there a target-allowlist concept (per-identity reachable target
|
||||
sets), and does it live in `AccessControl` or in the open handler's
|
||||
params validation?
|
||||
|
||||
**Status:** open — needs alkcall ADR-050 review + a survey of how
|
||||
alktty scoped its gate. Half-answer (hunch): scope-gate for the open
|
||||
plus an open-handler-level target policy hook; ownership for
|
||||
*registered/listened* tunnels (which are persistent resources), not for
|
||||
ephemeral dials.
|
||||
|
||||
### OQ-TN-09: Lifecycle, teardown, and error reporting
|
||||
|
||||
The two-pump shape gives byte-level teardown for free (EOF sentinels;
|
||||
channels drops per-channel senders on transport EOF — alknet ADR-078,
|
||||
POC issue #6). What's missing is the error/level above bytes:
|
||||
|
||||
- How does a failed target dial reach the consumer (e.g. "connection
|
||||
refused to 10.0.0.5:80")? Is there a structured error frame in the
|
||||
channel before close, a `channel/close` with reason, or call-level
|
||||
error on the open op?
|
||||
- Is there a "tunnel established/failed" ack before byte pumping starts
|
||||
(alktty has the negotiation frame; the POC's tunnel handler had
|
||||
nothing — it dialed and pumped)?
|
||||
- Half-open semantics: one direction EOFs, the other keeps pumping
|
||||
(standard two-pump behavior) — is that always desired, or does the
|
||||
consumer need a "close both" control?
|
||||
|
||||
**Status:** open — needs a wire-format decision (ADR) if an error frame
|
||||
is added. Half-answer (hunch): a self-contained control frame (alktty
|
||||
ADR-006 shape) carrying an establishment result/error, sent before any
|
||||
data chunk; dial errors are tunnel-closing (the whole channel dies),
|
||||
whereas byte-level EOFs stay per-direction.
|
||||
|
||||
### OQ-TN-10: POC scope for what remains unvalidated
|
||||
|
||||
The alknet-channels POC validated TCP only. Candidate targeted POCs
|
||||
Phase 0 may need (in rough priority order, per the SDD process's
|
||||
"validate promising approaches"):
|
||||
|
||||
1. **UDP tunnel POC** — boundary-preserving length framing over a
|
||||
channels channel, per-endpoint multiplexing inside one association,
|
||||
backpressure behavior. Derisks OQ-TN-02 (and OQ-TN-07's option B).
|
||||
2. **Reverse-flow POC** — `-R`-style: the accept side listens, the far
|
||||
side carries. Derisks OQ-TN-03's advertisement/lifecycle shape.
|
||||
3. **Unix socket + stdio bridge POC** — cheap; validates "substrate
|
||||
agnostic" beyond IP substrates.
|
||||
4. **Two-pump helper extraction spike** — OQ-TN-06, only after 1–3.
|
||||
|
||||
POCs live in `.worktrees/research/<task-id>/` per the SDD process, or as
|
||||
standalone crates (`/workspace/alknet-channels-poc` precedent).
|
||||
|
||||
**Status:** open — pick 1 (and probably 2) after the research pass;
|
||||
3 is cheap enough to fold into whichever POC runs first.
|
||||
|
||||
## Survey / prior-art list
|
||||
|
||||
Candidate reading for the research specialist (to be expanded):
|
||||
|
||||
- SSH channel/forwarding model: RFC 4254 §7 (direct-tcpip /
|
||||
forwarded-tcpip), OpenSSH `-L`/`-R`/`-D` semantics, russh's
|
||||
`ChannelOpen` framing (russh is already in `/workspace/russh`).
|
||||
- SOCKS5 (RFC 1928): addressing (ATYP), UDP ASSOCIATE framing,
|
||||
per-endpoint multiplexing — the closest standardized "arbitrary
|
||||
tunnel + UDP" model.
|
||||
- tun2proxy (`/workspace/tun2proxy`): UDP-over-TCP tunnel framing in
|
||||
production; also handles DNS over tunnel.
|
||||
- quinn-proxy-poc (`/workspace/quinn-proxy-poc`) and iroh
|
||||
(`/workspace/iroh`): datagram-native transports; how they model
|
||||
per-endpoint flows.
|
||||
- alknet docs: ADR-071 §ALPN table (`alknet/tunnel` row), ADR-078,
|
||||
`docs/architecture/crates/channels/channel-operations.md` (`params`
|
||||
for `alknet/tunnel` is "the target resource"), and the hub-relay
|
||||
interaction (ADR-042/079).
|
||||
- alktty: `NegotiateRequest` shape (self-contained negotiation
|
||||
precedent), `TtyBackend` inversion point, `TTY_OPEN_SCOPE` access
|
||||
gate.
|
||||
|
||||
## Convergence checklist (what Phase 0 must produce)
|
||||
|
||||
- [ ] Survey notes: SSH/SOCKS5/tun2proxy addressing + UDP framing
|
||||
(OQ-TN-01, OQ-TN-02)
|
||||
- [ ] Recommendation: addressing format sketch (OQ-TN-01)
|
||||
- [ ] Recommendation: datagram strategy (OQ-TN-02) + ALPN strategy
|
||||
dependent on it (OQ-TN-07)
|
||||
- [ ] Recommendation: direction model (-L/-R/-D) (OQ-TN-03) + API
|
||||
surface sketch satisfying no-forced-binding (OQ-TN-04)
|
||||
- [ ] Decision input: backend trait vs no-trait (OQ-TN-05)
|
||||
- [ ] Targeted POC(s) run + summary (OQ-TN-10) — UDP first, reverse
|
||||
flow second
|
||||
- [ ] Open questions promoted to Phase 1
|
||||
`docs/architecture/open-questions.md` with statuses
|
||||
Reference in New Issue
Block a user