alktunnels/docs/architecture/decisions/003-codec-and-udp-framing.md

ADR-003: Data-Plane Codec — Raw Pass-Through (Stream) / Mandatory Length Framing (UDP)

Status

Accepted (2026-09-07; supersedes the Phase 0 "codec direction" hunch by making the UDP framing mandatory — reverse POC finding F-2)

Context

The tunnel data plane rides a channels data channel: raw bytes inside the channel BiStream (the channels layer strips its 8-byte header transparently — alknet ADR-093/ADR-071; AGENTS.md convention 5). The tunnel protocol owns whatever framing it puts inside the BiStream. A tunnel has one data stream per direction, so there is no sub-demux key — a 5-byte header like alktty's (with stream_type) would be pure overhead here.

The Phase 0 direction (phase-0-findings.md checklist, 2026-09-06): raw pass-through for stream substrates (0 B tunnel overhead), [len: u16 BE] per datagram for UDP (2 B/datagram), with the empty-datagram (len=0) semantic left as a Phase 1 residue.

Two validation passes sharpened this:

1. Forward POC (2026-09-06, 17 tests): the codec round-trips datagrams split across chunks, batched in one chunk, partial headers, and empty datagrams (len=0 is a legal payload, NOT EOF — EOF is the channels-level length=0 chunk sentinel; the two coexist at different layers). The 1400-byte (max ethernet MTU payload) datagram rides the bounded-buffer path; >65535 is rejected at frame time.
2. Reverse POC finding F-2 (2026-09-07, executable-pinned): in the RAW pass-through pump shape, an empty UDP datagram is a zero-byte read from the substrate adapter — tokio::io::copy treats Ok(0) as end-of-stream and shuts the pump down. An empty datagram and the zero-length EOF sentinel are the SAME wire shape at the pump level. The test pins the non-round-trip so the conclusion is executable, not anecdotal.

Decision

The data plane inside a alk/tunnel channel is:

• Stream substrates (tcp, unix): raw pass-through. The halves are the tunnel; zero tunnel-level framing. The only wire overhead is the channels 8-byte chunk header. A zero-length read on a stream substrate is genuinely EOF (stream semantics) — no collision. Both pumps run alkcall::channels::pump_bidi (ADR-050) over the raw halves.
• Datagram substrate (udp): mandatory [len: u16 BE][datagram] framing, applied at the substrate/pump boundary. One datagram per length-prefixed frame, on BOTH directions:
  • The establisher wraps the dialed UDP socket in the framed adapter BEFORE returning the plan (the plan payload is the adapter, not a raw socket) — the pump handler stays substrate-agnostic (ADR-004's invariant; producer.md's placement).
  • Consumer side: the TunnelSession's datagram variant does the same over the adopted halves.
  • len=0 is a legal empty datagram (DNS-over-TCP-style zero-payload probes); it round-trips under the codec (forward POC). EOF is exclusively the channels-level sentinel; the codec never emits zero-length reads, so the layers never collide. This is the load-bearing property F-2 validated from a second angle: the length prefix makes an empty datagram two bytes — unambiguous with EOF.
• MTU discipline: the codec rejects framing a >65535-byte datagram (Oversize error at frame time, never a wire overflow — the u16 length field would wrap). A truncated receive (caller buffer smaller than the datagram) fails loudly at the adapter level — resolved 2026-09-07 (OQ-TN-13, fail-loud): silent truncation would corrupt the framing invariants (the same invariant class F-2 protects).
• Chunking is transparent. Datagrams split across channel chunks, batch into single chunks, and survive partial headers — the incremental decoder is the only consumer-visible decode path (forward POC: maximally awkward 7-byte chunk splits verified).

Consequences

• TCP/unix tunnels have 0 B tunnel overhead — the SSH direct-tcpip experience (the payload is the bytes).
• UDP correctness requires the codec — raw pass-through over UDP is structurally broken for empty datagrams (F-2). The substrate discriminator (ADR-001) is what tells both sides to engage the codec; a consumer bypassing it for UDP is a spec violation, not a degraded mode.
• tokio::io::copy-shaped pumps fit UDP only through the framed adapter (the UdpHalf + codec wrapper presents AsyncRead/AsyncWrite whose zero-length read never occurs for a well-formed stream — len=0 yields a 2-byte wire frame).
• The BAST document exists (AGENTS.md convention 12 — the mandatory framing IS binary framing beyond pass-through, so the trigger fired): docs/architecture/bast.md describes the [len: u16 BE] datagram frame per the BAST meta-schema; the hand-rolled codec remains the runtime implementation.
• No mid-stream control frames. Pump-phase failures are EOF-shaped by design (alkcall ADR-049 §6); there is no tunnel-level error frame, no KEEPALIVE (channels transport liveness is upstream's), no flow-expiry frame (producer-side bookkeeping).

References

• OQ-TN-02 (promoted), OQ-TN-13 (resolved by this ADR's fail-loud posture)
• Forward POC: docs/research/poc-summary.md §3 (codec decision + sentinel layering + MTU); reverse POC: docs/research/reverse-poc-summary.md §F-2 (the mandate)
• docs/research/ssh-socks5-survey.md §UDP (tun2proxy udpgw framing precedent — no CONN_ID, no FLAGS; boundary framing only)
• alknet ADR-071/093 (channels wire format; the zero-length sentinel)
• alkcall channels::pump_bidi (ADR-050)