docs: UDP tunnel POC summary — shape validated end-to-end

17 tests pass (7 codec + 10 integration). The producer/consumer
shape, ADR-049 establishment integration, codec decision, and typed
establishment errors all validated over alkcall 0.5.0 channels.

Key findings folded into the ledger:
- OQ-TN-05 resolved for stream substrates: no backend trait —
  pump_halves generic over boxed halves (TCP into_split, UDP
  UdpHalf adapter); remaining thread is only the hub re-produce
  composition question
- OQ-TN-06 convergence test satisfied: producer pump and consumer
  pump are the same shape modulo channel side; extraction decidable
  in Phase 1
- New finding for the alkcall ledger: OpenHandler JoinHandle
  semantics are load-bearing and undocumented — a handler that
  returns before the data plane starts tears the channel down at
  birth (the POC found this by hanging; the pump must be awaited
  inline). Plus: Establishment should carry the channel plan
  (the reserved extension point) to kill the establisher handoff
  side-channel
- UdpHalf truncation semantics flagged for the Phase 1 ADR
  (recommend fail-loud over silent truncation)

POC crate: /workspace/alktunnels-udp-poc (db224cc).
This commit is contained in:
2026-09-07 07:53:39 +00:00
parent 20672be48c
commit 6beabe9457
2 changed files with 351 additions and 8 deletions
+33 -8
View File
@@ -512,12 +512,15 @@ so the question:
surface as-is? If a helper is warranted, it may share shape with the
substrate dial/listen trait — which would argue for the trait.
**Status:** open — needs a survey of what backends would actually
implement (local TCP? docker exec? ssh -w?) before deciding trait vs
no-trait. Half-answer (hunch): a thin trait (or just a fn alias) for
"obtain a bidirectional substrate stream for a target," possibly no
trait at all if the only meaningful backends are local sockets — decide
after surveying candidate backends.
**Status: resolved for stream substrates, 2026-09-06 (POC).**
`pump_halves` is generic over `(AsyncRead, AsyncWrite)` halves — TCP
contributes `into_split()` halves, UDP contributes the `UdpHalf`
socket adapter, the pump never knows which. "Produce boxed halves for
a resource" is a function, not a trait; the `TunnelBackend` trait
question resolves to *no trait* for substrate access. The only
remaining thread is the hub re-produce composition question (above) —
a Phase 1 spec question, and now decidable against the validated
producer/consumer surface.
### OQ-TN-06: The two-pump helper — extract now?
@@ -539,6 +542,13 @@ consumer (POC tunnel was the first; SSH `direct-tcpip` would be a third).
not a blocker for the spec. Half-answer: the helper probably belongs
upstream (alkcall, near the channels-adapter handler-integration
conventions) but only if the two shapes genuinely converge.
**Convergence input 2026-09-06 (POC):** the producer pump
(`pump_halves`) and the consumer pump (`take_halves` + copy) are the
same shape modulo channel side — the ADR-078 convergence test is
satisfied; extraction is decidable in Phase 1. Also surfaced by the
POC: the `OpenHandler` JoinHandle semantics (see `poc-summary.md` §
Issues Surfaced #1) are where such a helper would pin the teardown
contract.
### OQ-TN-07: ALPN strategy
@@ -707,6 +717,10 @@ Phase 0 may need (in rough priority order, per the SDD process's
fit: chunk-size vs datagram-size interaction, MTU cap against the
channels bounded buffers, idle-expiry mapping, and the chosen
endpoint-addressing shape. Derisks OQ-TN-02 (and OQ-TN-07's option B).
**Done 2026-09-06** — see `poc-summary.md`. 17 tests pass; the
producer/consumer shape validated end-to-end over alkcall 0.5.0
(establisher + two-pump + codec + typed establishment errors).
2. **Reverse-flow POC**`-R`-style: the accept side listens, the far
side carries. Derisks OQ-TN-03's advertisement/lifecycle shape.
Template available: SSH's `tcpip-forward` global-request
@@ -807,7 +821,18 @@ Candidate reading for the research specialist (to be expanded):
OQ-TN-05 trait shape
- [ ] Decision input: backend trait vs no-trait (OQ-TN-05), now
including the hub re-produce composition question
- [ ] Targeted POC(s) run + summary (OQ-TN-10) — UDP first, reverse
flow second
- [ ] Targeted POC(s) run + summary (OQ-TN-10) — UDP POC DONE
(2026-09-06, `poc-summary.md`); reverse-flow POC remains
- [x] Two-pump helper convergence input gathered (OQ-TN-06) — the POC
gives both shapes (producer `pump_halves`, consumer
`take_halves` + copy): same shape modulo channel side; the
alknet ADR-078 convergence test is satisfied, extraction now
decidable in Phase 1
- [x] OQ-TN-05 executable input (backend trait vs no-trait) — POC
confirms stream substrates need no trait: `pump_halves` is
generic over boxed halves (TCP `into_split`, UDP `UdpHalf`
adapter); "produce halves for a resource" is a function. The
hub re-produce composition question is the only remaining
OQ-TN-05 thread
- [ ] Open questions promoted to Phase 1
`docs/architecture/open-questions.md` with statuses
+318
View File
@@ -0,0 +1,318 @@
---
status: complete
last_updated: 2026-09-06
---
# alktunnels: UDP Tunnel POC Research Summary
**Status:** Research complete — the producer/consumer shape validated
end-to-end over alkcall 0.5.0 channels, including the ADR-049
establishment phase and the codec decision (raw pass-through for TCP,
`[len: u16 BE]` for UDP). 17 tests pass (7 codec + 10 integration);
clippy `-D warnings` clean; fmt clean. The remaining unknowns are
spec-scope (Phase 1 ADRs), not feasibility.
**Date:** 2026-09-06
**Scope:** Validates the OQ-TN-10 #1 POC (UDP tunnel) — plus the
consumer-side of the establisher hand-off (ADR-049), the two-pump
pump-handler contract's `JoinHandle` semantics (a real finding — see
§Issues Surfaced), and the OQ-TN-05 "no backend trait for stream
substrates" judgment in executable form.
---
## Executive Summary
A POC (`alktunnels-udp-poc`, `/workspace/alktunnels-udp-poc`)
validated the alktunnels producer/consumer shape end-to-end:
1. **The producer shape**`channels/tunnel/sub` open op registered
via `ChannelCore::register_openable_with_establisher` (ADR-049):
the establisher validates params semantically (registry lookup of
`{resource, substrate}` → backing address, producer-side), dials
the substrate (TCP connect / UDP connect), and delivers the dialed
handle to the pump handler via a handoff; the wrapper replies only
after establishment. A refused dial is a typed
`channel:open_failed` call error with
`details.reason ∈ {unknown_resource, dial_failed, resource_shortage}`
— the SSH contract consumer-visibly, verified from the consumer
side (`TunnelSession::open` resolving `Err` with the reason).
2. **The two-pump data plane** — the pump handler accepts the
channel's yield-once `BiStream`, splits it, and runs two
`tokio::io::copy` pumps with shutdown-on-completion (alknet ADR-078)
against the substrate halves. 1 MiB round-trips through the
two-pump shape with the bounded-buffer backpressure path intact.
3. **The codec decision** — raw pass-through for TCP (zero
tunnel-level framing; the channels 8-byte header is the only
overhead) and `[len: u16 BE][datagram]` for UDP (2 bytes/datagram).
Datagram boundaries survive chunk splitting, batching, and empty
datagrams (`len=0`) are legal payloads, NOT EOF — the EOF sentinel
lives at the channels layer (`BiStream` EOF) and the two never
collide (different layers).
4. **The consumer shape**`TunnelSession::open`
`ChannelClient::open_channel` → split the channel `BiStream`
present raw halves (TCP) or framed datagrams (UDP). One session =
one channel = one tunnel; the channel ID is the flow key (no
conn-id — OQ-TN-02 resolution holds).
**17 tests pass** (7 wire codec + 10 integration: TCP echo, TCP 1 MiB,
UDP boundary preservation incl. empty datagrams, UDP-through-chunk-
splitting at the codec level, establisher unknown-resource +
dial-failure typed errors, scope-gate FORBIDDEN, TCP+UDP concurrent
channels on one connection). Clippy clean, fmt clean.
---
## What was built
```
alktunnels-udp-poc/
Cargo.toml — alkcall 0.5.0; tokio wasm-clean subset + net (POC-local)
src/
lib.rs — module docs; the POC overview
params.rs — TunnelParams {resource, substrate} (the OQ-TN-01 shape)
wire.rs — the codec: frame_datagram / DatagramReader
([len: u16 BE] framing; incremental decode across
chunk boundaries; len=0 = empty datagram, not EOF)
producer.rs — tunnel_open_spec (scope-gated, Sub-typed, channel_open
marker), tunnel_establisher (registry lookup + dial +
handoff), HandleHandoff (resource-keyed), pump_halves
(the ADR-078 two-pump shape, generic over halves),
make_tunnel_pump_handler (the OpenHandler),
register_tunnel_openable (the wiring), UdpHalf
(UdpSocket as AsyncRead/AsyncWrite halves)
consumer.rs — TunnelSession (typed client): open / stream_halves /
send_datagram / recv_datagram / take_halves;
establishment_reason (the ADR-049 typed-error branch)
harness.rs — wire_tunnel (the alktty testing.rs pattern: duplex
transport, ChannelsAdapter, install_channel_zero hook,
register_tunnel_openable on the hook's registry,
tunnel_identity with the tunnel:open scope)
tests/
tunnel_poc.rs — the 10 integration tests
```
The POC follows the `alknet-channels-poc` convention: standalone crate
in the workspace root, findings written back into the consuming
crate's `docs/research/`.
## Key findings
### 1. The OpenHandler JoinHandle contract — the pump handler MUST return the pump's own handle
The most important finding of the POC, and it was found by the POC
hanging, not by reading: **the `OpenHandler`'s returned `JoinHandle`
is the wrapper's teardown signal.** The wrapper (`run_open_wrapper`)
awaits it, then tears the channel down (drop the demux sender → EOF to
the handler's read half → the pumps exit). A handler that spawns the
pump as a *nested* task and returns the outer handle immediately
completes the wrapper's await at birth → teardown → both pumps see
instant EOF with zero bytes — every tunnel appears to "connect and
instantly close" while everything upstream looks healthy.
```rust
// WRONG — outer task returns immediately; wrapper tears down at birth:
tokio::spawn(async move {
let bidi = conn.accept_bi().await?;
tokio::spawn(pump_halves(bidi, read, write)); // fire-and-forget
// <- handler ends here; wrapper's raw_task.await resolves NOW
})
// RIGHT — the returned JoinHandle tracks the pump's lifetime:
tokio::spawn(async move {
let bidi = conn.accept_bi().await?;
let _ = pump_halves(bidi, read, write).await; // awaited inline
})
```
This is an API-sharpness finding for alkcall (the `OpenHandler`
contract's `JoinHandle` semantics are load-bearing and undocumented —
the type docs say "so the wrapper can record it for teardown (abort on
channel/close / connection drop)" but not that an early return *is*
an early teardown). Worth an upstream doc note; the shape itself is
correct. Filed as a follow-up for the alkcall review ledger.
The pump shape itself (ADR-078) is unchanged and correct: each pump
shuts down the opposite sink on completion; `tokio::join!` on the two
pumps; the wrapper's teardown on pump completion drops the opposite
direction's sender — half-open semantics fall out naturally (one
direction EOFs, the other keeps pumping until its own EOF).
### 2. The establisher hand-off works; its shape is POC-grade only
The ADR-049 amendment says the establisher gets `(input, auth)` only —
the channel's `BiStream` belongs to the pump handler. The POC's
`HandleHandoff` (a `Mutex<HashMap<resource, SubstrateHandle>>` + poll
loop in `take`) works because the wrapper *guarantees* ordering: the
establisher is awaited before the reply, and the pump handler is
spawned after. The delivery is therefore always-before-take.
But the resource-keyed map is a POC simplification that only works for
one open per resource at a time (concurrent opens of the same resource
would race the slot). The real crate's mechanism should be the
ADR-049 reserved extension point: **`Establishment` should carry the
channel plan** (the struct is already documented as "Reserved for a
channel plan — today the wrapper consults only success/failure"). When
alkcall extends `Establishment` to carry data, the handoff mechanism
dies entirely and the establisher returns the dialed handle in its
result. That is the recommended upstream evolution (small, additive —
`Establishment {}``Establishment { plan: Option<Value> }` or a
generic), and it removes the last awkwardness of the establisher
shape. Filed as a follow-up for the alkcall ledger.
### 3. The codec decision holds exactly as designed
- **TCP: raw pass-through.** The consumer writes bytes into the
channel halves; the producer's pump copies them to the TCP socket.
Zero tunnel-level framing; the only wire overhead is the channels
8-byte header per chunk. The 1 MiB test exercises bounded buffers
end-to-end with no deadlock (the alknet-channels POC's
`tunnel_large_payload` result reproduces on alkcall 0.5.0).
- **UDP: `[len: u16 BE]` framing.** `DatagramReader` is an
incremental decoder — datagrams split across chunks, datagrams
batched in one chunk, partial headers, and empty datagrams all
round-trip (verified at the codec level with maximally awkward
7-byte chunk splits, and end-to-end through the channels stack).
- **Sentinel layering verified:** `len == 0` in the codec is an empty
datagram (DNS-over-TCP-style; round-trips through a UDP echo);
EOF is the channels-level `length=0` chunk on the `BiStream`. The
two coexist without ambiguity — the consumer's `recv_datagram`
returns `None` only on stream EOF, never on an empty datagram.
- **MTU discipline:** the codec rejects >65535 at frame time; the
1400-byte (max ethernet MTU payload) datagram round-trips through
the bounded-buffer path. The E-04 64-parked-chunks bound was noted
as a sizing constraint; the POC's datagram sizes stay far below it
per channel.
### 4. The typed establishment contract is consumer-visible and complete
The consumer branches on `ChannelOpenError::CallFailed { error }`:
`error.code == "channel:open_failed"` with
`details.reason``{unknown_resource, dial_failed, resource_shortage}`
(from the establisher) — plus the wrapper-level `timeout` reason and
the pre-allocation failures (`FORBIDDEN` for the scope gate,
`channel:too_many_channels` for the cap). All verified in tests:
- unknown resource → `unknown_resource` (the "ghost" test)
- dial to a closed port → `dial_failed` (the "dead" test)
- scope-less identity → `FORBIDDEN` (the "mallory" test)
The SSH contract holds end-to-end: a failed open never returns a
`channel_id`; the consumer's session never half-exists.
### 5. Stream substrates need no backend trait (OQ-TN-05, executable confirmation)
`pump_halves` is generic over `(AsyncRead, AsyncWrite)` halves. TCP
contributes `OwnedReadHalf`/`OwnedWriteHalf` (`TcpStream::into_split`);
UDP contributes `UdpHalf` (the POC's `UdpSocket` adapter — `poll_recv`
as `AsyncRead`, `poll_send` as `AsyncWrite`, connected-socket
semantics). The pump never knows which. The "backend" question
collapses to "produce boxed halves for a resource" — a function, not a
trait, for everything the producer dials. The remaining OQ-TN-05
question (whether the *hub's* re-produce proxy needs a trait) is
unaffected by this finding; it was always about composition, not
substrate access.
## Issues surfaced (for the spec / upstream)
1. **`OpenHandler` JoinHandle semantics are undocumented and
load-bearing** (alkcall). The wrapper's teardown-on-handler-exit is
the intended contract, but nothing in the type docs warns that an
early return = teardown-at-birth. Recommend: a doc note on
`OpenHandler` and `ChannelCore::register_openable*` ("the returned
JoinHandle must track the data-plane lifetime; returning early
tears the channel down"), possibly a debug-level warning when the
wrapper's awaited handler exits without having accepted the
channel's `BiStream`. → alkcall review ledger.
2. **`Establishment` should carry the channel plan** (alkcall). The
struct is already documented as reserved for it. With `plan`
carrying the dialed handle (or any established state), the
establisher→handler handoff becomes a return value instead of a
side-channel. Unblocks concurrent same-resource opens and removes
the POC's poll-loop take. → alkcall review ledger (additive,
two-way door).
3. **`UdpHalf` truncation semantics** (alktunnels spec): `poll_recv`
into a smaller caller buffer truncates the datagram silently
(UDP semantics). For well-formed peers the codec's MTU bound makes
this unreachable; the Phase 1 spec should pin whether the real
crate's UDP adapter errors on truncation instead (fail loud) —
recommend erroring, since truncation would corrupt the length-
framed stream invariants. → Phase 1 ADR note.
4. **Consumer-side pump symmetry** (alktunnels Phase 1): the consumer
mirrors the producer's pump shape (`tokio::io::copy` between
local substrate and channel halves, shutdown-on-completion). The
POC's `TunnelSession::take_halves` exists precisely so the assembly
layer can pump generically — the OQ-TN-06 helper extraction
question can now be evaluated with both shapes in hand (producer
`pump_halves`, consumer `take_halves` + copy): they are the same
shape modulo channel side. The convergence test alknet ADR-078
asked for is satisfied.
## What the POC does NOT validate
1. **Real transports**`tokio::io::duplex` stands in for
TCP/TLS/QUIC. The alkcall layer owns transport; unchanged from the
alknet-channels POC scope.
2. **Unix-socket and stdio substrates** — the OQ-TN-10 #3 "cheap"
item. The pump is substrate-agnostic by construction (finding #5);
these are wiring, not mechanics. Fold into the real crate's test
suite rather than a separate POC.
3. **The reverse-flow (`-R`) advertisement** — OQ-TN-10 #2 remains
open; the SSH `tcpip-forward` template in the survey is the
lifecycle shape to validate there.
4. **Relay traversal** — hub relay (alkcall ADR-042) forwards data
channels byte-for-byte; the POC's single-hop shape is what
relays see per leg. The terminate-and-re-produce hub proxy (the
hub model) is an assembly-layer concern.
5. **Multi-endpoint UDP gateway resources** — the udpgw-shaped
framing-inside-a-channel variant (OQ-TN-02's "split by path"
dynamic half). Not a v1 base-protocol concern; the codec keeps the
door open (framing inside datagram payloads, self-describing).
6. **Wasm** — the POC uses `tokio/net` for real sockets; the
wasm-clean story is for the real crate's protocol-only core (the
codec and pump are pure byte/async work; `UdpHalf` and the dial
paths are the feature-gated parts). Not exercised here.
## Test coverage
```
running 7 tests (src/wire.rs)
test wire::tests::datagram_split_across_chunks ... ok
test wire::tests::empty_datagram_round_trips_and_is_not_eof ... ok
test wire::tests::frames_and_parses_single_datagram ... ok
test wire::tests::oversize_datagram_is_rejected_at_frame_time ... ok
test wire::tests::partial_header_waits ... ok
test wire::tests::mid_datagram_state_is_observable ... ok
test wire::tests::two_datagrams_in_one_chunk ... ok
running 10 tests (tests/tunnel_poc.rs)
test tcp_tunnel_echo_round_trip ... ok
test tcp_tunnel_large_payload_round_trip ... ok (1 MiB)
test udp_tunnel_round_trip_preserves_boundaries ... ok (incl. empty dg)
test udp_empty_datagram_round_trips ... ok
test udp_datagrams_survive_chunk_splitting ... ok (100 dgs, 7-byte chunks)
test establisher_failure_is_typed_open_failed_no_phantom_channel ... ok
test establisher_dial_failure_reports_dial_failed ... ok
test open_without_scope_is_forbidden ... ok
test udp_large_datagram_through_bounded_buffers ... ok (1400 B)
test udp_and_tcp_concurrent ... ok
```
## References
- alkcall 0.5.0 — ADR-049 (establishment phase),
`ChannelCore::register_openable_with_establisher`,
`OpenEstablisher`/`Establishment`/`EstablishmentError`,
`ChannelOpenError` (typed client error), `run_open_wrapper`
(`src/channels/operations.rs`).
- alknet ADR-078 — the two-pump shutdown-on-completion contract;
`pump_halves` is its direct implementation.
- alktunnels `docs/research/phase-0-findings.md` — the OQ ledger this
POC closes items against (OQ-TN-01 params shape, OQ-TN-02 split by
path, OQ-TN-05 halves-not-trait, OQ-TN-09 ADR-049 supersession,
OQ-TN-10 #1).
- `docs/research/ssh-socks5-survey.md` — the reason-code vocabulary
the establisher errors surface as.
- alktty `src/testing.rs` — the harness pattern this POC's
`wire_tunnel` mirrors.
- tun2proxy `src/udpgw.rs` — the framing precedent the UDP codec
improves on (no CONN_ID, no FLAGS; boundary framing only).