alktunnels/docs/research/poc-summary.md

status, last_updated

status	last_updated
complete	2026-09-07

alktunnels: UDP Tunnel POC Research Summary

Status: Research complete — the producer/consumer shape validated end-to-end over alkcall 0.5.0 channels, including the ADR-049 establishment phase and the codec decision (raw pass-through for TCP, [len: u16 BE] for UDP). 17 tests pass (7 codec + 10 integration); clippy -D warnings clean; fmt clean. The remaining unknowns are spec-scope (Phase 1 ADRs), not feasibility. Both upstream findings below landed in alkcall 0.6.0 (review 007) — see §Resolutions 2026-09-07. Date: 2026-09-06 (resolutions added 2026-09-07) Scope: Validates the OQ-TN-10 #1 POC (UDP tunnel) — plus the consumer-side of the establisher hand-off (ADR-049), the two-pump pump-handler contract's JoinHandle semantics (a real finding — see §Issues Surfaced), and the OQ-TN-05 "no backend trait for stream substrates" judgment in executable form.

---

Executive Summary

A POC (alktunnels-udp-poc, /workspace/alktunnels-udp-poc) validated the alktunnels producer/consumer shape end-to-end:

1. The producer shape — channels/tunnel/sub open op registered via ChannelCore::register_openable_with_establisher (ADR-049): the establisher validates params semantically (registry lookup of {resource, substrate} → backing address, producer-side), dials the substrate (TCP connect / UDP connect), and delivers the dialed handle to the pump handler via a handoff; the wrapper replies only after establishment. A refused dial is a typed channel:open_failed call error with details.reason ∈ {unknown_resource, dial_failed, resource_shortage} — the SSH contract consumer-visibly, verified from the consumer side (TunnelSession::open resolving Err with the reason).
2. The two-pump data plane — the pump handler accepts the channel's yield-once BiStream, splits it, and runs two tokio::io::copy pumps with shutdown-on-completion (alknet ADR-078) against the substrate halves. 1 MiB round-trips through the two-pump shape with the bounded-buffer backpressure path intact.
3. The codec decision — raw pass-through for TCP (zero tunnel-level framing; the channels 8-byte header is the only overhead) and [len: u16 BE][datagram] for UDP (2 bytes/datagram). Datagram boundaries survive chunk splitting, batching, and empty datagrams (len=0) are legal payloads, NOT EOF — the EOF sentinel lives at the channels layer (BiStream EOF) and the two never collide (different layers).
4. The consumer shape — TunnelSession::open → ChannelClient::open_channel → split the channel BiStream → present raw halves (TCP) or framed datagrams (UDP). One session = one channel = one tunnel; the channel ID is the flow key (no conn-id — OQ-TN-02 resolution holds).

17 tests pass (7 wire codec + 10 integration: TCP echo, TCP 1 MiB, UDP boundary preservation incl. empty datagrams, UDP-through-chunk- splitting at the codec level, establisher unknown-resource + dial-failure typed errors, scope-gate FORBIDDEN, TCP+UDP concurrent channels on one connection). Clippy clean, fmt clean.

---

What was built

alktunnels-udp-poc/
  Cargo.toml        — alkcall 0.5.0; tokio wasm-clean subset + net (POC-local)
  src/
    lib.rs          — module docs; the POC overview
    params.rs       — TunnelParams {resource, substrate} (the OQ-TN-01 shape)
    wire.rs         — the codec: frame_datagram / DatagramReader
                      ([len: u16 BE] framing; incremental decode across
                      chunk boundaries; len=0 = empty datagram, not EOF)
    producer.rs     — tunnel_open_spec (scope-gated, Sub-typed, channel_open
                      marker), tunnel_establisher (registry lookup + dial +
                      handoff), HandleHandoff (resource-keyed), pump_halves
                      (the ADR-078 two-pump shape, generic over halves),
                      make_tunnel_pump_handler (the OpenHandler),
                      register_tunnel_openable (the wiring), UdpHalf
                      (UdpSocket as AsyncRead/AsyncWrite halves)
    consumer.rs     — TunnelSession (typed client): open / stream_halves /
                      send_datagram / recv_datagram / take_halves;
                      establishment_reason (the ADR-049 typed-error branch)
    harness.rs      — wire_tunnel (the alktty testing.rs pattern: duplex
                      transport, ChannelsAdapter, install_channel_zero hook,
                      register_tunnel_openable on the hook's registry,
                      tunnel_identity with the tunnel:open scope)
  tests/
    tunnel_poc.rs   — the 10 integration tests

The POC follows the alknet-channels-poc convention: standalone crate in the workspace root, findings written back into the consuming crate's docs/research/.

Key findings

1. The OpenHandler JoinHandle contract — the pump handler MUST return the pump's own handle

The most important finding of the POC, and it was found by the POC hanging, not by reading: the OpenHandler's returned JoinHandle is the wrapper's teardown signal. The wrapper (run_open_wrapper) awaits it, then tears the channel down (drop the demux sender → EOF to the handler's read half → the pumps exit). A handler that spawns the pump as a nested task and returns the outer handle immediately completes the wrapper's await at birth → teardown → both pumps see instant EOF with zero bytes — every tunnel appears to "connect and instantly close" while everything upstream looks healthy.

// WRONG — outer task returns immediately; wrapper tears down at birth:
tokio::spawn(async move {
    let bidi = conn.accept_bi().await?;
    tokio::spawn(pump_halves(bidi, read, write)); // fire-and-forget
    // <- handler ends here; wrapper's raw_task.await resolves NOW
})

// RIGHT — the returned JoinHandle tracks the pump's lifetime:
tokio::spawn(async move {
    let bidi = conn.accept_bi().await?;
    let _ = pump_halves(bidi, read, write).await; // awaited inline
})

This is an API-sharpness finding for alkcall (the OpenHandler contract's JoinHandle semantics are load-bearing and undocumented — the type docs say "so the wrapper can record it for teardown (abort on channel/close / connection drop)" but not that an early return is an early teardown). Worth an upstream doc note; the shape itself is correct. Filed as a follow-up for the alkcall review ledger.

The pump shape itself (ADR-078) is unchanged and correct: each pump shuts down the opposite sink on completion; tokio::join! on the two pumps; the wrapper's teardown on pump completion drops the opposite direction's sender — half-open semantics fall out naturally (one direction EOFs, the other keeps pumping until its own EOF).

2. The establisher hand-off works; its shape is POC-grade only

The ADR-049 amendment says the establisher gets (input, auth) only — the channel's BiStream belongs to the pump handler. The POC's HandleHandoff (a Mutex<HashMap<resource, SubstrateHandle>> + poll loop in take) works because the wrapper guarantees ordering: the establisher is awaited before the reply, and the pump handler is spawned after. The delivery is therefore always-before-take.

But the resource-keyed map is a POC simplification that only works for one open per resource at a time (concurrent opens of the same resource would race the slot). The real crate's mechanism should be the ADR-049 reserved extension point: Establishment should carry the channel plan (the struct is already documented as "Reserved for a channel plan — today the wrapper consults only success/failure"). When alkcall extends Establishment to carry data, the handoff mechanism dies entirely and the establisher returns the dialed handle in its result. That is the recommended upstream evolution (small, additive — Establishment {} → Establishment { plan: Option<Value> } or a generic), and it removes the last awkwardness of the establisher shape. Filed as a follow-up for the alkcall ledger.

3. The codec decision holds exactly as designed

• TCP: raw pass-through. The consumer writes bytes into the channel halves; the producer's pump copies them to the TCP socket. Zero tunnel-level framing; the only wire overhead is the channels 8-byte header per chunk. The 1 MiB test exercises bounded buffers end-to-end with no deadlock (the alknet-channels POC's tunnel_large_payload result reproduces on alkcall 0.5.0).
• UDP: [len: u16 BE] framing. DatagramReader is an incremental decoder — datagrams split across chunks, datagrams batched in one chunk, partial headers, and empty datagrams all round-trip (verified at the codec level with maximally awkward 7-byte chunk splits, and end-to-end through the channels stack).
• Sentinel layering verified: len == 0 in the codec is an empty datagram (DNS-over-TCP-style; round-trips through a UDP echo); EOF is the channels-level length=0 chunk on the BiStream. The two coexist without ambiguity — the consumer's recv_datagram returns None only on stream EOF, never on an empty datagram. Mandate strengthened 2026-09-07 (reverse POC F-2): without the codec (raw pass-through), an empty datagram IS a zero-byte read — colliding with EOF at the pump level. UDP must ride the codec.
• MTU discipline: the codec rejects >65535 at frame time; the 1400-byte (max ethernet MTU payload) datagram round-trips through the bounded-buffer path. The E-04 64-parked-chunks bound was noted as a sizing constraint; the POC's datagram sizes stay far below it per channel.

4. The typed establishment contract is consumer-visible and complete

The consumer branches on ChannelOpenError::CallFailed { error }: error.code == "channel:open_failed" with details.reason ∈ {unknown_resource, dial_failed, resource_shortage} (from the establisher) — plus the wrapper-level timeout reason and the pre-allocation failures (FORBIDDEN for the scope gate, channel:too_many_channels for the cap). All verified in tests:

• unknown resource → unknown_resource (the "ghost" test)
• dial to a closed port → dial_failed (the "dead" test)
• scope-less identity → FORBIDDEN (the "mallory" test)

The SSH contract holds end-to-end: a failed open never returns a channel_id; the consumer's session never half-exists.

5. Stream substrates need no backend trait (OQ-TN-05, executable confirmation)

pump_halves is generic over (AsyncRead, AsyncWrite) halves. TCP contributes OwnedReadHalf/OwnedWriteHalf (TcpStream::into_split); UDP contributes UdpHalf (the POC's UdpSocket adapter — poll_recv as AsyncRead, poll_send as AsyncWrite, connected-socket semantics). The pump never knows which. The "backend" question collapses to "produce boxed halves for a resource" — a function, not a trait, for everything the producer dials. The remaining OQ-TN-05 question (whether the hub's re-produce proxy needs a trait) is unaffected by this finding; it was always about composition, not substrate access.

Issues surfaced (for the spec / upstream)

1. OpenHandler JoinHandle semantics are undocumented and load-bearing (alkcall). The wrapper's teardown-on-handler-exit is the intended contract, but nothing in the type docs warns that an early return = teardown-at-birth. Recommend: a doc note on OpenHandler and ChannelCore::register_openable* ("the returned JoinHandle must track the data-plane lifetime; returning early tears the channel down"), possibly a debug-level warning when the wrapper's awaited handler exits without having accepted the channel's BiStream. → alkcall review ledger.
2. Establishment should carry the channel plan (alkcall). The struct is already documented as reserved for it. With plan carrying the dialed handle (or any established state), the establisher→handler handoff becomes a return value instead of a side-channel. Unblocks concurrent same-resource opens and removes the POC's poll-loop take. → alkcall review ledger (additive, two-way door).
3. UdpHalf truncation semantics (alktunnels spec): poll_recv into a smaller caller buffer truncates the datagram silently (UDP semantics). For well-formed peers the codec's MTU bound makes this unreachable; the Phase 1 spec should pin whether the real crate's UDP adapter errors on truncation instead (fail loud) — recommend erroring, since truncation would corrupt the length- framed stream invariants. → Phase 1 ADR note.
4. Consumer-side pump symmetry (alktunnels Phase 1): the consumer mirrors the producer's pump shape (tokio::io::copy between local substrate and channel halves, shutdown-on-completion). The POC's TunnelSession::take_halves exists precisely so the assembly layer can pump generically — the OQ-TN-06 helper extraction question can now be evaluated with both shapes in hand (producer pump_halves, consumer take_halves + copy): they are the same shape modulo channel side. The convergence test alknet ADR-078 asked for is satisfied.

Resolutions 2026-09-07 (alkcall 0.6.0, review 007)

Both upstream findings above landed, with deviations recorded in the review and ADRs:

1. JoinHandle semantics (→ R-02) — documented on the OpenHandler type and both register_openable* methods: the returned JoinHandle must track the data-plane lifetime; the wrapper awaits it and its completion triggers channel teardown. Plus the POC's second ask: a yield-once acceptance flag in ChannelBidiStreamSource and a debug! birth-teardown hint in the teardown task, so the hang-this-POC-found shape is diagnosable at runtime, not just in docs.
2. Establishment carries the plan (→ R-01) — filled as Establishment { plan: Option<ChannelPlan> } with Establishment::new(plan); typed-opaque (ChannelPlan = Arc<dyn Any + Send + Sync>), not the Option<Value> sketch here — the payloads are live handles (dialed sockets, TTY handles) with no JSON representation. The OpenHandler gains a plan parameter and the wrapper threads it. The POC's HandleHandoff side-channel and its same-resource race are dead: the establisher returns the dialed handle directly (Ok(Establishment::new(dialed))). Confirmed under concurrency 2026-09-07 by the reverse-flow POC (reverse-poc-summary.md): two concurrent opens of the same resource, distinct channels, distinct dialed handles — the race this handoff had is structurally gone. (F-1 there: plan payloads must be Send + Sync.)
3. Two-pump helper (→ R-03, bonus) — alkcall::channels::pump_bidi pinned upstream (ADR-050), with (u64, u64) copy counts (no io::Result — errors are EOF-shaped per the ADR-078 contract). The POC's pump_halves shape converged and became the helper; Phase 1 consumes it instead of hand-rolling.

What the POC does NOT validate

1. Real transports — tokio::io::duplex stands in for TCP/TLS/QUIC. The alkcall layer owns transport; unchanged from the alknet-channels POC scope.
2. Unix-socket and stdio substrates — the OQ-TN-10 #3 "cheap" item. The pump is substrate-agnostic by construction (finding #5); these are wiring, not mechanics. Fold into the real crate's test suite rather than a separate POC.
3. The reverse-flow (-R) advertisement — OQ-TN-10 #2 remains open; the SSH tcpip-forward template in the survey is the lifecycle shape to validate there.
4. Relay traversal — hub relay (alkcall ADR-042) forwards data channels byte-for-byte; the POC's single-hop shape is what relays see per leg. The terminate-and-re-produce hub proxy (the hub model) is an assembly-layer concern.
5. Multi-endpoint UDP gateway resources — the udpgw-shaped framing-inside-a-channel variant (OQ-TN-02's "split by path" dynamic half). Not a v1 base-protocol concern; the codec keeps the door open (framing inside datagram payloads, self-describing).
6. Wasm — the POC uses tokio/net for real sockets; the wasm-clean story is for the real crate's protocol-only core (the codec and pump are pure byte/async work; UdpHalf and the dial paths are the feature-gated parts). Not exercised here.

Test coverage

running 7 tests (src/wire.rs)
test wire::tests::datagram_split_across_chunks ... ok
test wire::tests::empty_datagram_round_trips_and_is_not_eof ... ok
test wire::tests::frames_and_parses_single_datagram ... ok
test wire::tests::oversize_datagram_is_rejected_at_frame_time ... ok
test wire::tests::partial_header_waits ... ok
test wire::tests::mid_datagram_state_is_observable ... ok
test wire::tests::two_datagrams_in_one_chunk ... ok

running 10 tests (tests/tunnel_poc.rs)
test tcp_tunnel_echo_round_trip ... ok
test tcp_tunnel_large_payload_round_trip ... ok          (1 MiB)
test udp_tunnel_round_trip_preserves_boundaries ... ok   (incl. empty dg)
test udp_empty_datagram_round_trips ... ok
test udp_datagrams_survive_chunk_splitting ... ok        (100 dgs, 7-byte chunks)
test establisher_failure_is_typed_open_failed_no_phantom_channel ... ok
test establisher_dial_failure_reports_dial_failed ... ok
test open_without_scope_is_forbidden ... ok
test udp_large_datagram_through_bounded_buffers ... ok   (1400 B)
test udp_and_tcp_concurrent ... ok

References

• alkcall 0.5.0 — ADR-049 (establishment phase), ChannelCore::register_openable_with_establisher, OpenEstablisher/Establishment/EstablishmentError, ChannelOpenError (typed client error), run_open_wrapper (src/channels/operations.rs). 0.6.0 supersedes parts: ADR-049 amendment 2 (Establishment.plan), ADR-050 (pump_bidi), review 007 (filed from this POC, all units landed).
• alknet ADR-078 — the two-pump shutdown-on-completion contract; pump_halves is its direct implementation.
• alktunnels docs/research/phase-0-findings.md — the OQ ledger this POC closes items against (OQ-TN-01 params shape, OQ-TN-02 split by path, OQ-TN-05 halves-not-trait, OQ-TN-09 ADR-049 supersession, OQ-TN-10 #1).
• docs/research/ssh-socks5-survey.md — the reason-code vocabulary the establisher errors surface as.
• alktty src/testing.rs — the harness pattern this POC's wire_tunnel mirrors.
• tun2proxy src/udpgw.rs — the framing precedent the UDP codec improves on (no CONN_ID, no FLAGS; boundary framing only).