glm-5.3-flash
648feb194d
Worktree/branch for repo-dependent POCs; standalone crate in the global workspace for self-contained POCs (alknet-channels-poc precedent); findings always land in docs/research/.
688 lines
37 KiB
Markdown
688 lines
37 KiB
Markdown
---
|
||
status: draft
|
||
last_updated: 2026-09-05
|
||
---
|
||
|
||
# alktunnels — Phase 0 Research Findings
|
||
|
||
This document captures Phase 0 (Exploration) findings and open design
|
||
questions for the `alktunnels` crate. The objective of Phase 0 per
|
||
`docs/sdd_process.md` is: *"Capture vision and guiding principles; research
|
||
options; validate approaches; converge on a recommended approach."* It is the
|
||
input to Phase 1 (Architecture), where the Architect will produce
|
||
`docs/architecture/` specs, ADRs, and open questions.
|
||
|
||
Drafted 2026-09-05, emerging from the initial setup discussion. The crate is
|
||
the sibling of alktty (`alk/tty` — terminal sessions) on the alkcall
|
||
substrate: where alktty multiplexes one service with a fixed five-stream
|
||
channel structure, alktunnels generalizes the tunnel handler shape to
|
||
arbitrary bidirectional tunnels in the `ssh -L` / `ssh -D` sense — TCP, UDP,
|
||
unix sockets, and other stream or datagram substrates.
|
||
|
||
The 2026-09-05 first revision adds the tun2proxy UDP gateway prior art
|
||
(`OQ-TN-01`, `OQ-TN-02`, `OQ-TN-07`, `OQ-TN-10`) and the transport story
|
||
clarification (TCP vs QUIC at the channels layer, invisible to this crate —
|
||
§What is already settled). This prior art was discussed with the POC agent
|
||
around the alknet-channels POC but never documented; it is captured here so
|
||
it survives into the spec.
|
||
|
||
## What is already settled
|
||
|
||
The foundation is POC-validated and ADR-pinned; this crate is not starting
|
||
from zero. It inherits:
|
||
|
||
- **The demux→Connection→handler→mux path** — validated by the
|
||
alknet-channels POC (Target 3), now production alkcall channels. The
|
||
tunnel payload is raw bytes inside a channels data channel; channels
|
||
strips its 8-byte header transparently (alknet ADR-093 / alkcall
|
||
ADR-035).
|
||
- **The two-pump handler shape** — one pump per direction, each pump MUST
|
||
shut down the opposite sink on completion (`try_join!` alone deadlocks;
|
||
alknet ADR-078). POC-validated with a 1 MiB backpressure test. This
|
||
crate is the *second* two-pump consumer the ADR deferred helper
|
||
extraction for (the first was the POC's tunnel handler; SSH
|
||
`direct-tcpip` would be a later third).
|
||
- **The producer/consumer model** — producer registers openable channels
|
||
via `ChannelCore::register_openable` (authorization for free via
|
||
`AccessControl`); consumer opens tunnel channels via `ChannelClient`
|
||
(alkcall ADR-037, ADR-043). Connection direction is independent of
|
||
tunnel direction.
|
||
- **The backend inversion point pattern** — substrate-specific types
|
||
(`TcpStream`, `UdpSocket`, unix sockets) confined to feature-gated
|
||
backend modules, injected at the assembly layer, never imported from
|
||
the shared/producer/consumer modules (alktty `TtyBackend` precedent).
|
||
- **The wasm-clean default crate** — protocol-only code compiles to
|
||
`wasm32-unknown-unknown`; socket/platform I/O is feature-gated
|
||
(alktty precedent).
|
||
- **The relay story** — tunnels traverse alkcall hub relays
|
||
transparently via byte-for-byte data-channel forwarding with ID
|
||
rewrite (alkcall ADR-042). No tunnel-specific relay work.
|
||
- **The transport story** — the underlying channels transport is TCP or
|
||
QUIC (QUIC preferred) but that is the alkcall layer's concern, not
|
||
this crate's: the tunnel protocol sees a `BiStream` and is
|
||
transport-agnostic like the sibling crates. UDP tunneling (below)
|
||
rides the same chunk stream; the stream-vs-datagram question is
|
||
about what the tunnel protocol frames *inside* the channel, not
|
||
about the transport.
|
||
|
||
## Prior art: the hub-owns-the-connection model (`OQ-TN-01`, `OQ-TN-03`, `OQ-TN-04`, `OQ-TN-08`)
|
||
|
||
Discussed 2026-09-05. The standing stance across the alk* crates: **the
|
||
hub owns the connection and explicitly proxies to expose resources for
|
||
others.** A produced resource is not a socket — it is an ACL-scoped
|
||
*virtual resource* on the side that can reach the real target. Applied
|
||
to tunnels, this dissolves the SSH direction model:
|
||
|
||
- **Role follows the resource.** Whoever can reach the target is the
|
||
producer (registers openable tunnel channels via
|
||
`ChannelCore::register_openable`, dials/serves the target); whoever
|
||
wants the bytes is the consumer (opens channels via `ChannelClient`).
|
||
This holds for both SSH `-L` and `-R`: in both, the entry-point side
|
||
opens the channel (consumer) and the target side handles the open
|
||
(producer) — the only difference is which machine hosts the entry
|
||
point, which is assembly-layer wiring, not protocol.
|
||
- **The "exposed port" is a virtual port.** The far side may *think* it
|
||
exposed a port on the remote end; what actually exists is an
|
||
ACL-scoped resource in the connection/identity registry — the same
|
||
shape as operation registration, not a bind. Example: a self-hosted
|
||
gitea (HTTP in a docker container, really binding a port) is tunneled
|
||
to the hub; the hub proxies the tunnel to consumers whose ACL grants
|
||
access to the gitea HTTP service. A SOCKS5 service is the same shape,
|
||
further downstream.
|
||
- **The hub proxy is a producer wrapping a consumer.** The proxy is an
|
||
assembly-layer construct. On leg 1 (gitea side ↔ hub): gitea side is
|
||
producer, hub is consumer. On leg 2 (hub ↔ service consumer): the hub
|
||
re-produces the resource ACL-scoped — producer wrapping its leg-1
|
||
consumer role. Data path per hop is the same two-pump shape; the
|
||
consumer opens a channel naming the resource, the producer's open
|
||
handler dials the target. This means the hub *terminates and
|
||
re-produces* rather than relaying byte-for-byte — unlike alkcall
|
||
ADR-042's transparent data-channel relay — because ACL must apply
|
||
per hop. (ADR-042 relays remain valid for transports that don't
|
||
need per-hop ACL; the hub's tunnel proxy is the explicit-ACL path.)
|
||
- **`-D` (dynamic/SOCKS) composes.** A SOCKS5 server at some assembly
|
||
layer is just a consumer that opens channels with per-connection
|
||
dynamic targets, gated by the producer's target policy. Not a base
|
||
crate concern (same conclusion as the original half-answer, now with
|
||
the mechanism named).
|
||
- **No forced binding, everywhere (OQ-TN-04 resolved).** Binding is
|
||
always assembly-layer and optional, on either side. The protocol
|
||
never binds: a producer "produces a resource" (which may or may not
|
||
correspond to a local bind — a docker container's port, a unix
|
||
socket, an in-process service); a consumer "consumes it." The SSH
|
||
mental image of "expose a port" is an illusion the assembly layer
|
||
may create locally; the protocol only ever carries
|
||
produce/consume bookkeeping.
|
||
|
||
**Model, one sentence:** a producer produces a resource (a stream of
|
||
basically anything, if TCP and UDP are supported); a consumer consumes
|
||
it; direction, bind, and ACL scoping are assembly-layer concerns
|
||
wrapped around that pair.
|
||
|
||
**Residue (folded into other OQs):**
|
||
- How does a consumer *discover* produced resources — resolved
|
||
2026-09-05: the existing bidirectional ACL-filtered ops listing
|
||
(openable channels are operations, alkcall ADR-047). See OQ-TN-08.
|
||
- Dynamic-target policy for `-D`-style opens — dissolved 2026-09-05:
|
||
`-D` is "just tunnel a socks5 connection"; target selection lives in
|
||
the socks5 protocol at the producing side, governed by the same
|
||
op-level ACL. See OQ-TN-08.
|
||
- Whether the hub proxy needs anything from this crate (a
|
||
re-produce helper? typed open-handler composition?) or composes from
|
||
the public producer/consumer surface as-is → spec question for
|
||
Phase 1, tracked here as part of OQ-TN-05.
|
||
|
||
## Prior art: tun2proxy UDP gateway (`OQ-TN-01`, `OQ-TN-02`, `OQ-TN-07`, `OQ-TN-10`)
|
||
|
||
`/workspace/tun2proxy/src/udpgw.rs` implements a UDP gateway over a TCP
|
||
stream — structurally the same problem this crate faces for UDP tunnels
|
||
over channels data channels. Discussed with the alknet-channels POC
|
||
agent as the example of "UDP over a stream substrate," but never
|
||
documented. Key mechanics, all of which generalize:
|
||
|
||
- **Per-datagram length framing over the stream** — the packet format is
|
||
`LEN(u16 BE) | FLAGS(u8) | CONN_ID(u16) | [SOCKS5 address] | DATA`
|
||
(`udpgw.rs:82-88`). Boundary preservation is re-added by the protocol,
|
||
not by the substrate: exactly the "length-prefix each datagram inside
|
||
the channel" half-answer in OQ-TN-02, proven in production.
|
||
- **SOCKS5 address format travels per data packet** — `ATYP`
|
||
(0x01 IPv4 / 0x03 domain / 0x04 IPv6) + variable address + port
|
||
(`udpgw.rs:68-76`). This is concrete prior art for OQ-TN-01's
|
||
addressing: a scheme-tagged addressing encoding with v4/v6/domain
|
||
coverage already standardized. Note the asymmetry with TCP tunnels:
|
||
for UDP, the remote endpoint is per-datagram, not per-channel.
|
||
- **One stream carries many UDP flows** — `CONN_ID(u16)` multiplexes
|
||
associations over a single gateway connection (`udpgw.rs:66`),
|
||
with `keepalive` (0x01) and `error` (0x20) flag packets as the only
|
||
non-data frame types (`udpgw.rs:21-26`). This is the "one channel =
|
||
one association, per-endpoint multiplexing inside" half-answer in
|
||
OQ-TN-02, with the refinement that the per-endpoint multiplexing key
|
||
(`CONN_ID`) is protocol-level, allocated by the client, u16.
|
||
- **Flow lifecycle is packet-level** — `udp_timeout` idle expiry,
|
||
`keepalive_time` heartbeats on idle connections
|
||
(`UDPGW_KEEPALIVE_TIME = 30s`, `udpgw.rs:16`), and an MTU cap
|
||
(`parse_udp_response` rejects `data.len() > udp_mtu`,
|
||
`udpgw.rs:527`). Also `UDPGW_MAX_CONNECTIONS = 5` pooled gateway
|
||
connections *above* the packet layer — a throughput choice, not a
|
||
protocol requirement; channels gives us N channels already.
|
||
- **Implications for alktunnels:**
|
||
- UDP boundary preservation over the chunk stream is validated
|
||
prior art, not speculation — raises confidence in the OQ-TN-02
|
||
half-answer considerably.
|
||
- The frame-type set (DATA/KEEPALIVE/ERR) is a useful minimal
|
||
vocabulary — it maps onto OQ-TN-09's establishment/error frame
|
||
question (tun2proxy uses flag packets, alktty uses typed control
|
||
chunks; both are self-contained frames inside the data stream).
|
||
- Per-datagram addressing (SOCKS5-style) vs per-channel addressing
|
||
(fixed target at open) is a real fork for the params design: TCP
|
||
tunnels fix the target at open; UDP associations may either fix
|
||
one endpoint at open or carry per-datagram addresses like udpgw.
|
||
This interaction is unresolved and feeds OQ-TN-01 + OQ-TN-07.
|
||
- NAT/keepalive concerns partially disappear on channels: the
|
||
underlying transport (QUIC/TCP) handles connection keepalive, and
|
||
channel liveness is the channels layer's concern. The tunnel
|
||
protocol likely needs only flow-level idle expiry, not
|
||
transport-level keepalive packets — TBD in the spec.
|
||
|
||
## Prior art: the alktty stream-splitting pattern (`OQ-TN-09`, `OQ-TN-05`)
|
||
|
||
Discussed 2026-09-05. alktty tunnels io-like streams: the adapter
|
||
splits the `BiStream` from `accept_bi` into read/write halves
|
||
(`tokio::io::split`), wraps each half in a `ChunkReader`/`ChunkWriter`
|
||
(`wire.rs` — the 5-byte `[stream_type: u8][length: u32 be][payload]`
|
||
codec), and pumps chunks between the wire halves and the backend's
|
||
`TtyHandle` halves. Strip packets on the way in, wrap packets on the
|
||
way out; everything above the codec sees plain `AsyncRead`/`AsyncWrite`.
|
||
|
||
The proposal: that same pattern (or close) may be *the* tunnel pattern.
|
||
A tunnel is then literally "tty without the five stream types" — split
|
||
the channel's `BiStream`, wrap the halves in a codec, pump to/from
|
||
substrate halves. What differs per concern is only the codec and the
|
||
far-end handle:
|
||
|
||
| | alktty | alktunnels (TCP) | alktunnels (UDP) |
|
||
|---|---|---|---|
|
||
| wire codec | 5-byte, 5 stream types | raw pass-through or minimal framing | length-prefixed datagrams (tun2proxy-style) |
|
||
| far end | `TtyHandle` (stdin/stdout/stderr/ctrl) | `TcpStream` split halves | `UdpSocket` + flow table |
|
||
| pumps | 3 (stdout/stderr + stdin, exit future coordinates) | 2 (the ADR-078 shape) | 2 + flow expiry |
|
||
|
||
Implications:
|
||
|
||
- **OQ-TN-09 falls out naturally.** alktty's `STREAM_CTRL_IN/OUT`
|
||
(flag frames inside the same codec) is the precedent for the
|
||
establishment/error frame: same codec, one extra stream type
|
||
(`TUNNEL_CTRL` or similar), typed control messages — rather than a
|
||
separate framing layer. alktty's zero-length-sentinel EOF semantics
|
||
carry over unchanged.
|
||
- **OQ-TN-05 (backend trait) gets a concrete shape to judge.** If
|
||
tunnels are "split + wrap + pump," the far-end handle is just
|
||
"(AsyncRead + AsyncWrite) halves" for stream substrates — which is
|
||
either a very thin trait or no trait (fn returning boxed halves).
|
||
The UDP case's flow table is the real differentiator to design
|
||
around, not the stream case.
|
||
- **Possible convergence with OQ-TN-06 (two-pump helper).** If the
|
||
pump loop shape is identical to alktty's modulo the number of pumps,
|
||
a shared helper extraction (this crate's second-consumer moment,
|
||
alknet ADR-078) should be evaluated against alktty's `pump_session`
|
||
too — the convergence test the ADR asked for may be nearly free.
|
||
- **Wasm story intact** — the codec is pure byte manipulation; the
|
||
split/pump plumbing is tokio-async, no substrate types in the
|
||
protocol layer. Same as alktty.
|
||
|
||
Open: whether the tunnel codec is *literally* alktty's 5-byte format
|
||
with different stream-type constants (reuse), a stripped 5-byte
|
||
variant (1 stream type + ctrl), or raw pass-through with framing only
|
||
for UDP (the OQ-TN-01 reframe suggests TCP needs nothing beyond the
|
||
channels chunk stream). Phase 1 ADR, before the first consumer.
|
||
|
||
## Open Questions
|
||
|
||
These are the design questions Phase 0 must resolve (or explicitly defer)
|
||
before the architecture spec. They are numbered OQ-TN-01.. so they can be
|
||
referenced, tracked, and promoted into `docs/architecture/open-questions.md`
|
||
in Phase 1. Half-answers and hunches are marked as such — the point of this
|
||
document is to hold them without forcing premature decisions.
|
||
|
||
### OQ-TN-01: Target addressing format
|
||
|
||
What does the tunnel `params` on `channel/open` look like? alknet ADR-071
|
||
§ALPN table noted `alknet/tunnel` as `[0, 1]` data in/out only, but the
|
||
addressing scheme was never decided. It must cover at minimum:
|
||
|
||
- TCP dial (`host:port`)
|
||
- UDP (associate-style or endpoint-style — see OQ-TN-02)
|
||
- Unix domain sockets (path)
|
||
- Direction (who dials the target — see OQ-TN-03, resolved)
|
||
- Bind/listen vs dial semantics (see OQ-TN-04, mostly resolved)
|
||
|
||
Considerations:
|
||
|
||
- `params` is ALPN-specific JSON, interpreted by the open handler, not by
|
||
the channels layer (alknet ADR-075 / alkcall ADR-039). alktty's
|
||
precedent is the `NegotiateRequest` shape — a self-contained JSON
|
||
object carried in the open op.
|
||
- The addressing string is wire-stable once a consumer exists (one-way
|
||
door). It must be substrate-extensible without format changes (a new
|
||
substrate should be an additive `scheme` value, not a v2 format).
|
||
- Prior art to survey: SSH forwarding models (`direct-tcpip`,
|
||
`forwarded-tcpip`), SOCKS5 addressing (ATYP + addr + port), russh
|
||
(`/workspace/russh` — note: russh has no UDP channel type at all, so
|
||
there is no `direct-udp`-style prior art there; still useful for
|
||
channel-open framing and addressing-intel generally).
|
||
- ALPN convention check (2026-09-05): the ALPN prefix swap `alknet/` →
|
||
`alk/` happened alkcall v0.1.1, before the first published consumer.
|
||
Everything in this crate's docs referencing `alknet/tunnel` means
|
||
`alk/tunnel` — a small thing, but docs/ADRs must not perpetuate the
|
||
old prefix or consumers will bake it in.
|
||
|
||
**Reframe 2026-09-05 — this is partly an XY problem.** The rich
|
||
"remote addressing" framing was chasing the wrong thing. Except in the
|
||
`-D`/dynamic case (which composes at the assembly layer), a tunnel is
|
||
either TCP or UDP, and `params` need only *identify a produced
|
||
resource* — not carry a general-purpose address. The producer owns
|
||
where the resource comes from (a local port, a docker container's
|
||
port, an in-process service, a unix socket — its problem, most likely
|
||
a local port). So `params` reduce to:
|
||
|
||
1. Which resource (a produced resource identifier — stable name or
|
||
target address, still to decide),
|
||
2. Substrate discriminator (`tcp` / `udp` / extensible) — so UDP is
|
||
structurally supported from day one even if the vast majority of
|
||
use is TCP.
|
||
|
||
Rich in-band addressing (SOCKS5 ATYP, per-datagram remote addresses)
|
||
enters only through the `-D`/dynamic-target composition path
|
||
(OQ-TN-03/OQ-TN-08 residue), not through the base open-op params.
|
||
|
||
**Status:** mostly resolved 2026-09-05 (pending the resource-naming
|
||
and discovery residue). Direction of travel:
|
||
- `params` = self-contained JSON object in the open op (alktty
|
||
`NegotiateRequest` precedent) — accepted path.
|
||
- `params` identify a produced resource + substrate discriminator;
|
||
producer owns the backing. No URL-style general addressing.
|
||
- Residue: stable resource name vs target address as the identifier
|
||
shape; discovery resolved 2026-09-05 — see OQ-TN-08 (the existing
|
||
ACL-filtered ops listing IS tunnel-resource discovery, since
|
||
openable channels are operations per alkcall ADR-047); exact JSON
|
||
field layout (Phase 1 spec, ADR before first consumer — wire-stable
|
||
once published).
|
||
|
||
### OQ-TN-02: Datagram substrates (UDP) — boundary preservation
|
||
|
||
Does a UDP tunnel preserve datagram boundaries end-to-end, or does the
|
||
tunnel present a byte-stream abstraction to the consumer (boundaries lost,
|
||
re-chunked arbitrarily)?
|
||
|
||
- Channels is a chunk stream with bounded buffers; the zero-length chunk
|
||
is the EOF sentinel — datagram boundaries are *not* preserved by the
|
||
substrate (alknet ADR-071/093; the POC only exercised TCP).
|
||
- SSH's `-D` UDP associate tunnels UDP as a stream with per-datagram
|
||
framing re-added by the tunnel protocol (e.g. SOCKS5 UDP over TCP).
|
||
Note (2026-09-05): russh does not support UDP channels at all — no
|
||
`direct-udp`-style prior art exists there; the tun2proxy gateway
|
||
(§Prior art) is the strongest framing precedent.
|
||
- iroh and quinn-proxy-poc have native datagram transports; tun2proxy has
|
||
a full UDP-over-TCP model worth reading.
|
||
- Boundary preservation is a wire-format decision (per-datagram length
|
||
framing inside the `BiStream`) and would need an ADR + possibly a BAST
|
||
document (AGENTS.md convention 12). Boundary loss is cheaper but
|
||
changes what protocols can ride the tunnel (DNS? QUIC? game traffic?).
|
||
- Datagrams also raise multiplexing questions TCP does not: one UDP
|
||
"association" carries many remote endpoints — does one tunnel channel
|
||
carry one endpoint or many, and how are per-endpoint replies routed?
|
||
|
||
**Status:** open — survey mostly resolved by tun2proxy prior art
|
||
(§Prior art): per-datagram length framing over the stream is
|
||
production-proven (`LEN | FLAGS | CONN_ID | [addr] | DATA`), one
|
||
stream carries many UDP flows via a protocol-level `CONN_ID`, and
|
||
flow lifecycle (idle timeout + keepalive) is packet-level. Remaining:
|
||
whether alktunnels fixes the UDP endpoint at open (per-channel, TCP-
|
||
like) or carries per-datagram addresses (udpgw-like), and whether a
|
||
u16 conn-id vocabulary is right for channels (vs the channel ID
|
||
itself doing the demux and one channel per UDP flow). Note (2026-09-05,
|
||
hub model §Prior art): if UDP resources are produced like any other
|
||
resource, endpoint-at-open aligns naturally with resource naming
|
||
(OQ-TN-01); per-datagram addressing matches the `-D`/dynamic-target
|
||
composition path instead. A targeted POC (OQ-TN-10 #1) is likely still
|
||
+EV for the chosen shape.
|
||
|
||
### OQ-TN-03: Direction semantics (`-L` / `-R` / dynamic)
|
||
|
||
**Status: resolved 2026-09-05** by the hub-owns-the-connection model
|
||
(§Prior art: the hub-owns-the-connection model). There is no
|
||
protocol-level direction: role follows the resource. Whoever can reach
|
||
the target is the producer (registers openable channels); whoever wants
|
||
the bytes is the consumer (opens channels). SSH `-L` and `-R` are the
|
||
same producer/consumer pair with the entry point on different machines —
|
||
assembly-layer wiring, not protocol. The "exposed port" is a virtual,
|
||
ACL-scoped resource on the producing side; the hub proxy is a producer
|
||
wrapping a consumer. `-D`/SOCKS composes as a consumer opening channels
|
||
with per-connection dynamic targets, gated by the producer's target
|
||
policy (OQ-TN-08) — not a base-crate concern. Original question retained
|
||
below for context.
|
||
|
||
Considerations (original):
|
||
|
||
SSH has three forwarding flavors; the crate must model them without
|
||
"server/client" framing:
|
||
|
||
- `-L` (local forward): consumer dials a local port; producer dials the
|
||
target. Channels flows consumer→producer; target dial happens on the
|
||
producer side. This is the POC's shape.
|
||
- `-R` (remote forward): producer (or a third party) listens; the
|
||
*consumer's* side dials or accepts incoming connections and asks the
|
||
other side to carry them. Channels flows producer→consumer.
|
||
- `-D` (dynamic/SOCKS): one side runs a SOCKS5 server; the target is
|
||
chosen per-connection by the client. Addressing arrives per-channel,
|
||
not per-tunnel-registration.
|
||
|
||
Both sides can be producer and consumer simultaneously (alkcall ADR-022/037
|
||
direction semantics), so the model must not bake direction into the
|
||
connection. The open questions:
|
||
|
||
- Is direction a field in `params`, or two distinct open-handler shapes /
|
||
ALPNs?
|
||
- How does `-R` register availability (the side that will carry traffic
|
||
advertises listen targets)? Does it interact with `channel/open` at all,
|
||
or is it a call-level operation ("please open a tunnel channel to me
|
||
when a local accept happens")?
|
||
- Dynamic (-D) may not be a tunnel concern at all — it may compose as
|
||
"SOCKS5 server implemented over alktunnels dial primitives" in a
|
||
separate crate. Keep or cut for v1?
|
||
|
||
**Status:** open — needs architecture decision. Half-answer (hunch): `-L`
|
||
is the channel/open handler; `-R` needs a small advertisement/lifecycle
|
||
surface; `-D` composes on top and is out of scope for the base crate.
|
||
|
||
### OQ-TN-04: No forced local binding
|
||
|
||
A tunnel must not require the producer (or consumer) to bind a local port.
|
||
The POC's shape dialed a target from the handler; binding is optional and
|
||
belongs to the caller (assembly layer), not the protocol crate. The API
|
||
surface must support:
|
||
|
||
- Dial flows with no local bind (POC shape) — covered.
|
||
- Listen flows where the binding happens on one side only.
|
||
- Unbound/abstract flows (e.g. unix socketpair-style, stdio bridges,
|
||
in-process pipes) where neither side binds.
|
||
|
||
The protocol layer must express "carry bytes between this target and this
|
||
channel" without assuming either endpoint is a bound socket. Substrate
|
||
modules (behind feature flags) own actual `bind()` calls; the protocol
|
||
owns bookkeeping only.
|
||
|
||
**Status:** open — mostly resolved (2026-09-05, see §Prior art: the
|
||
hub-owns-the-connection model): binding is always assembly-layer and
|
||
optional, on either side; the protocol never binds. What remains is the
|
||
concrete API surface — who calls what to start a tunnel in each mode
|
||
(produce-with-dial, produce-without-dial/accept-style, consume). This is
|
||
now a spec-shape task, not a research question.
|
||
|
||
### OQ-TN-05: Backend inversion point — is there a `TunnelBackend` trait?
|
||
|
||
alktty has `TtyBackend` because backends (local PTY, docker, SSH) produce
|
||
handles and the adapter pumps them. For tunnels, the producer side's
|
||
substrate action is narrower — dial a target, or accept on a listener —
|
||
so the question:
|
||
|
||
- Is a `TunnelBackend`-style trait needed at all, or is the two-pump
|
||
handler + feature-gated substrate modules (dial/listen helpers) the
|
||
whole story, with the assembly layer wiring substrate streams directly?
|
||
- If a trait: what is the handle type? A tunnel "handle" is just an
|
||
`AsyncRead + AsyncWrite` stream (or a datagram endpoint) — much thinner
|
||
than `TtyHandle`'s stdin/stdout/stderr/exit-code quadruple. The trait
|
||
may collapse to "produce a boxed stream for this target" plus a
|
||
listener variant.
|
||
- Backpressure/limits come from channels (AGENTS.md convention 10); the
|
||
backend trait must not add a second layer of them.
|
||
- New sub-question from the hub model (2026-09-05, §Prior art): the
|
||
hub's tunnel proxy *re-produces* a resource it consumes (producer
|
||
wrapping a consumer). Does the proxy need a composition helper from
|
||
this crate, or does it assemble from the public producer/consumer
|
||
surface as-is? If a helper is warranted, it may share shape with the
|
||
substrate dial/listen trait — which would argue for the trait.
|
||
|
||
**Status:** open — needs a survey of what backends would actually
|
||
implement (local TCP? docker exec? ssh -w?) before deciding trait vs
|
||
no-trait. Half-answer (hunch): a thin trait (or just a fn alias) for
|
||
"obtain a bidirectional substrate stream for a target," possibly no
|
||
trait at all if the only meaningful backends are local sockets — decide
|
||
after surveying candidate backends.
|
||
|
||
### OQ-TN-06: The two-pump helper — extract now?
|
||
|
||
alknet ADR-078 deferred helper extraction until a second two-pump consumer
|
||
exists ("a genuine deferral... the contract is decided (shutdown-on-
|
||
completion), only the extraction is deferred"). This crate is that second
|
||
consumer (POC tunnel was the first; SSH `direct-tcpip` would be a third).
|
||
|
||
- Does the helper live here (as a pub utility other handler crates can
|
||
use), or upstream in alkcall (which already owns `core` types)?
|
||
- Shape: `pump_bidi(recv, send) -> (Future, Future)` returning both
|
||
pumps with the shutdown-on-completion wired in? Or a
|
||
`join_two_pumps(a, b)` combinator?
|
||
- alknet ADR-057 (two-pump helper extraction OQ) noted the helper from
|
||
one consumer would bake in a wrong shape; with two consumers the shapes
|
||
should be compared before extraction.
|
||
|
||
**Status:** open — decide when the first real tunnel handler is written;
|
||
not a blocker for the spec. Half-answer: the helper probably belongs
|
||
upstream (alkcall, near the channels-adapter handler-integration
|
||
conventions) but only if the two shapes genuinely converge.
|
||
|
||
### OQ-TN-07: ALPN strategy
|
||
|
||
This crate owns the `alk/tunnel`-family ALPN(s). alkcall ADR-004: one
|
||
ALPN per protocol; `alk/` prefix. If stream (TCP/unix) and datagram (UDP)
|
||
tunnels get distinct ALPNs, the split must be decided before the first
|
||
consumer — ALPN strings are wire-stable once published.
|
||
|
||
- Option A: single `alk/tunnel` ALPN; substrate is a `params` field
|
||
(and datagram framing, if any, is self-describing inside the channel).
|
||
- Option B: `alk/tunnel` (stream) + `alk/tunnel-dgram` (datagram), so
|
||
the wire framing differs per ALPN cleanly.
|
||
- Channels' `params` is ALPN-specific, and the open-handler registry
|
||
dispatches per ALPN — both options are cheap mechanically; the cost is
|
||
consumer-side API bifurcation (two session types vs one with a
|
||
substrate enum).
|
||
|
||
**Status:** open — needs the OQ-TN-02 outcome first (if datagrams need
|
||
different framing, option B gets stronger). Note from the tun2proxy
|
||
prior art (§Prior art): udpgw runs its packet framing over a plain TCP
|
||
stream — one framing covers both the stream and datagram cases there.
|
||
If alktunnels follows the same shape (datagram framing self-describing
|
||
inside the channel), option A (single `alk/tunnel` ALPN) stays viable
|
||
even with UDP support; option B remains cleaner if the datagram
|
||
channel needs structurally different framing from the first chunk on.
|
||
|
||
### OQ-TN-08: Access control and ownership scope
|
||
|
||
Tunnels reach local networks — the open gate is the security boundary.
|
||
Shape follows alktty: `TUNNEL_OPEN_SCOPE` scope-gate, and the channels
|
||
path gets `AccessControl` wiring for free via
|
||
`ChannelCore::register_openable`. Open sub-questions:
|
||
|
||
- Should ownership (`OwnershipProvider.owns(...)`) be consulted for
|
||
tunnel targets, and what is the resource identity of a tunnel target
|
||
(a `host:port`? a registered tunnel name?), given targets may be
|
||
arbitrary strings and wildcard targets (`0.0.0.0/0`-style egress) may
|
||
be intentionally allowed for some identities?
|
||
- Is there a target-allowlist concept (per-identity reachable target
|
||
sets), and does it live in `AccessControl` or in the open handler's
|
||
params validation?
|
||
|
||
**Status: mostly resolved 2026-09-05** — this OQ pointed at a general
|
||
conceptual tangle ("the host owns the resource and proxies on top of
|
||
that" — how does that manifest for tunnels?), and the resolution is
|
||
the same posture alktty already uses:
|
||
|
||
- **Assume the resource is owned by the other side of the connection.**
|
||
At these protocol-crate levels (alktunnels, like alktty), the
|
||
protocol works under the assumption that a produced resource belongs
|
||
to the far side — so the ACL story is exactly alkcall's existing
|
||
op-level ACL. No new policy layer, no target allowlists, no
|
||
tunnel-specific ownership machinery. The actual proxy/overlay
|
||
mechanism (hub workers connecting in and exposing tunnels, the hub
|
||
providing an overlay to those resources based on the other side's
|
||
ACL) is a *downstream* (assembly-layer) concern.
|
||
- **Discovery is the existing bidirectional ops listing.** alkcall
|
||
already has an underlying bidirectional discovery mechanism — each
|
||
side can obtain the list of ops available to it (ACL-filtered). Since
|
||
openable channels *are* operations (alkcall ADR-047), that listing
|
||
*is* tunnel-resource discovery: a consumer asks "what ops are
|
||
available to me" and produced tunnel resources appear there, scoped
|
||
by identity. This is exactly how a consumer learns a socks5 tunnel,
|
||
a postgres TCP tunnel, or a redis tunnel is an available resource.
|
||
Examples of the pattern: services typically served over a VPN or
|
||
SSH tunnels (postgres, redis, gitea HTTP) — workers connect to a
|
||
hub, expose those tunnels, and the hub proxies them per-ACL.
|
||
- **`-D` simplifies to "just tunnel a socks5 connection."** The socks5
|
||
server lives on the producing side; the consumer opens an ordinary
|
||
tunnel channel to that resource and speaks socks5 inside it. Target
|
||
selection happens in the socks5 protocol at the far side — *not* in
|
||
tunnel params — so the previously-tracked "dynamic-target policy
|
||
hook for `-D`-style opens" residue dissolves: whatever ACL governs
|
||
the socks5 resource governs everything reachable through it, plus
|
||
whatever policy the socks5 implementation itself applies downstream.
|
||
|
||
Residue for Phase 1 (spec-shape, not research): confirm the
|
||
ops-listing surface carries enough per-resource metadata (substrate
|
||
type, resource name/description) for a consumer UI to distinguish
|
||
produced tunnels — or whether `params`-describing metadata rides
|
||
alongside the operation spec. That is an alkcall ADR-047 interaction,
|
||
not a new mechanism.
|
||
|
||
**Upstream posture (2026-09-05):** we own the upstream. Both prior
|
||
downstream crates (alktty, alkhttp — `/workspace/@alkdev/alkhttp`)
|
||
required fixes/additions to alkcall, and the rule is to make those
|
||
asks *early*: there are only three downstream dependents right now,
|
||
and each issue resolved upstream makes the next dependent cheaper. So
|
||
if the ops listing needs per-resource metadata (or anything else in
|
||
the ADR-047 interaction), the expectation is a small upstream alkcall
|
||
change in Phase 0/1 — not a workaround in this crate.
|
||
|
||
### OQ-TN-09: Lifecycle, teardown, and error reporting
|
||
|
||
The two-pump shape gives byte-level teardown for free (EOF sentinels;
|
||
channels drops per-channel senders on transport EOF — alknet ADR-078,
|
||
POC issue #6). What's missing is the error/level above bytes:
|
||
|
||
- How does a failed target dial reach the consumer (e.g. "connection
|
||
refused to 10.0.0.5:80")? Is there a structured error frame in the
|
||
channel before close, a `channel/close` with reason, or call-level
|
||
error on the open op?
|
||
- Is there a "tunnel established/failed" ack before byte pumping starts
|
||
(alktty has the negotiation frame; the POC's tunnel handler had
|
||
nothing — it dialed and pumped)?
|
||
- Half-open semantics: one direction EOFs, the other keeps pumping
|
||
(standard two-pump behavior) — is that always desired, or does the
|
||
consumer need a "close both" control?
|
||
|
||
**Status: direction set 2026-09-05** — the original half-answer is
|
||
accepted: a self-contained control frame (alktty ADR-006 shape)
|
||
carrying an establishment result/error, sent before any data chunk;
|
||
dial errors are tunnel-closing (the whole channel dies), whereas
|
||
byte-level EOFs stay per-direction. Beyond that, the frame vocabulary
|
||
should follow the alktty stream-splitting pattern (§Prior art: the
|
||
alktty stream-splitting pattern) rather than invent a parallel
|
||
mechanism. Concrete frame set lands as a Phase 1 ADR (wire-stable
|
||
before the first consumer).
|
||
|
||
### OQ-TN-10: POC scope for what remains unvalidated
|
||
|
||
The alknet-channels POC validated TCP only. Candidate targeted POCs
|
||
Phase 0 may need (in rough priority order, per the SDD process's
|
||
"validate promising approaches"):
|
||
|
||
1. **UDP tunnel POC** — boundary-preserving length framing over a
|
||
channels channel, per-endpoint multiplexing inside one association,
|
||
backpressure behavior. Partially derisked by the tun2proxy prior
|
||
art (§Prior art) — the POC now mainly validates *channels-layer*
|
||
fit: chunk-size vs datagram-size interaction, MTU cap against the
|
||
channels bounded buffers, idle-expiry mapping, and the chosen
|
||
endpoint-addressing shape. Derisks OQ-TN-02 (and OQ-TN-07's option B).
|
||
2. **Reverse-flow POC** — `-R`-style: the accept side listens, the far
|
||
side carries. Derisks OQ-TN-03's advertisement/lifecycle shape.
|
||
3. **Unix socket + stdio bridge POC** — cheap; validates "substrate
|
||
agnostic" beyond IP substrates.
|
||
4. **Two-pump helper extraction spike** — OQ-TN-06, only after 1–3.
|
||
|
||
POC placement conventions (2026-09-05): a POC that needs code from
|
||
this repo runs in a worktree/branch (`.worktrees/research/<task-id>/`
|
||
per the SDD process); a POC that is self-contained runs as a
|
||
standalone crate in the global workspace (the
|
||
`/workspace/alknet-channels-poc` precedent) with its findings written
|
||
into `docs/research/` here. Both are valid; pick per the POC's
|
||
dependency footprint. Findings always land in `docs/research/`
|
||
regardless of where the code lives.
|
||
|
||
**Status:** open — pick 1 (and probably 2) after the research pass;
|
||
3 is cheap enough to fold into whichever POC runs first.
|
||
|
||
## Survey / prior-art list
|
||
|
||
Candidate reading for the research specialist (to be expanded):
|
||
|
||
- SSH channel/forwarding model: RFC 4254 §7 (direct-tcpip /
|
||
forwarded-tcpip), OpenSSH `-L`/`-R`/`-D` semantics, russh's
|
||
`ChannelOpen` framing (russh is already in `/workspace/russh`;
|
||
confirmed 2026-09-05: no UDP channel support at all — no UDP prior
|
||
art there, but useful for open-op framing intel).
|
||
- SOCKS5 (RFC 1928): addressing (ATYP), UDP ASSOCIATE framing,
|
||
per-endpoint multiplexing — the closest standardized "arbitrary
|
||
tunnel + UDP" model. Relevant to the `-D` composition path
|
||
(OQ-TN-03 residue), not the base open-op params (OQ-TN-01 reframe).
|
||
- tun2proxy (`/workspace/tun2proxy`, `src/udpgw.rs`): UDP gateway over
|
||
TCP — per-datagram length framing, SOCKS5 per-datagram addressing,
|
||
CONN_ID flow multiplexing, keepalive/ERR flag packets, MTU cap,
|
||
idle expiry. Analyzed in §Prior art. Its `socks.rs` /
|
||
`proxy_handler.rs` are also relevant for the `-D` (dynamic/SOCKS)
|
||
composition question (OQ-TN-03).
|
||
- quinn-proxy-poc (`/workspace/quinn-proxy-poc`) and iroh
|
||
(`/workspace/iroh`): datagram-native transports; how they model
|
||
per-endpoint flows.
|
||
- alknet docs: ADR-071 §ALPN table (`alknet/tunnel` row), ADR-078,
|
||
`docs/architecture/crates/channels/channel-operations.md` (`params`
|
||
for `alknet/tunnel` is "the target resource"), and the hub-relay
|
||
interaction (ADR-042/079).
|
||
- alktty: `NegotiateRequest` shape (self-contained negotiation
|
||
precedent), `TtyBackend` inversion point, `TTY_OPEN_SCOPE` access
|
||
gate.
|
||
|
||
## Convergence checklist (what Phase 0 must produce)
|
||
|
||
- [ ] Survey notes: SSH/SOCKS5 addressing + UDP framing
|
||
(OQ-TN-01, OQ-TN-02) — tun2proxy UDP gateway done (§Prior art);
|
||
SSH/SOCKS5 survey delegated to research specialist
|
||
- [x] Reframe landed (OQ-TN-01) — params = self-contained JSON open-op
|
||
object identifying a produced resource + substrate
|
||
discriminator (`tcp`/`udp`/extensible); producer owns the
|
||
backing; no URL-style general addressing. Residue: resource
|
||
naming shape, exact JSON layout (Phase 1 ADR); discovery
|
||
resolved (OQ-TN-08)
|
||
- [x] Direction model resolved (OQ-TN-03) — hub-owns-the-connection
|
||
model: role follows the resource, no protocol-level direction;
|
||
`-D` composes at the assembly layer
|
||
- [x] No-forced-binding requirement encoded (OQ-TN-04) — binding is
|
||
always assembly-layer and optional; remaining work is the
|
||
concrete produce/consume API surface sketch (spec task)
|
||
- [x] Access control + discovery resolved (OQ-TN-08) — resource
|
||
ownership assumed on the far side; alkcall's existing ACL
|
||
applies as-is; ops listing (ADR-047) is tunnel-resource
|
||
discovery; `-D` = "tunnel a socks5 connection" (target selection
|
||
in the socks5 protocol, not params). Residue: per-resource
|
||
metadata in the ops listing — small upstream alkcall ask,
|
||
early per the upstream posture (alktty/alkhttp precedent)
|
||
- [x] Lifecycle/error direction set (OQ-TN-09) — establishment/error
|
||
control frame accepted in principle (dial errors are
|
||
channel-closing); frame vocabulary follows the alktty
|
||
stream-splitting pattern; concrete set = Phase 1 ADR
|
||
- [ ] Codec decision (new, from the alktty pattern prior art) — reuse
|
||
5-byte format vs stripped variant vs raw pass-through + UDP
|
||
framing; feeds OQ-TN-07 (ALPN) and the OQ-TN-05 trait shape
|
||
- [ ] Decision input: backend trait vs no-trait (OQ-TN-05), now
|
||
including the hub re-produce composition question
|
||
- [ ] Targeted POC(s) run + summary (OQ-TN-10) — UDP first, reverse
|
||
flow second
|
||
- [ ] Open questions promoted to Phase 1
|
||
`docs/architecture/open-questions.md` with statuses |