alktunnels/docs/architecture/wire.md

status, last_updated

status	last_updated
draft	2026-09-07

alktunnels — Wire Format

The alk/tunnel wire surface. Two layers: the open op (JSON params on the channels call plane — structured, schema-validated, typed errors) and the data plane (bytes/datagrams inside the channel BiStream — the codec, ADR-003). The channels layer strips its own 8-byte header transparently (alknet ADR-093); the tunnel protocol owns everything inside the BiStream.

Design decisions live in the ADRs (001, 002, 003); this document is the normative WHAT.

The Open Op (channels/tunnel/sub)

Sub-typed operation (alkcall ADR-047): the reply carries { "channel_id": <u32> } once; the data plane flows on the channel's BiStream — never in the call response stream.

Input (params)

{
  "resource": "postgres-primary",
  "substrate": "tcp"
}

• resource: string, required — the producer's stable resource name (NOT an address; the producer's registry maps it to its backing).
• substrate: string, required, enum ["tcp", "udp", "unix"] — the extensible discriminator (ADR-001); selects the data-plane framing. The schema ships with all three values in v1 (unix has an implementation path — OQ-TN-14). An older producer rejecting a newer substrate value is the SSH "unknown channel type" posture — loud, not silent.
• input_schema pins both as required strings; unknown substrate values fail schema validation (typed error — loud, not silent).
• Extensibility: a new substrate is a new enum value, not a format change. An older producer rejecting a newer substrate is the SSH "unknown channel type" posture.

Reply / errors

• Success: { "channel_id": <u32> } — the channel is open, the establishment phase (dial) succeeded, the pump handler is running.
• Failure: a typed channel:open_failed call error with details: { reason, message } — reason ∈ dial_failed / unknown_resource / resource_shortage / handler_error / timeout (alkcall ADR-049 §3). The registry's ACL gate runs before the wrapper (FORBIDDEN for a scope-less identity); the 256-channel cap is channel:too_many_channels. A failed open never returns a channel_id — no phantom channel (the SSH contract, POC-verified from the consumer side).
• The scope gate: the open op's AccessControl.required_scopes is ["tunnel:open"] (the TUNNEL_OPEN_SCOPE constant). Caller identity resolves in the alkcall 0.7.0 precedence order: payload auth_token > ServingConfig.identity > transport identity (CF-005); the establisher and pump handler receive the per-call opener identity (CF-006).

The Data Plane

One data stream per direction on one channel — the channel ID is the flow key (no conn-id, no sub-demux). Two codecs by substrate (ADR-003):

Stream substrates (tcp, unix): raw pass-through

The halves are the tunnel. Zero tunnel-level framing; the only wire overhead is the channels 8-byte chunk header. A zero-length read is genuinely EOF (stream semantics).

Both pumps run alkcall::channels::pump_bidi(channel, peer_read, peer_write) — the two-pump contract (alknet ADR-078, pinned upstream ADR-050): when one direction's source EOFs, the pump shuts the OPPOSITE sink down (the EOF sentinel crosses the mux; the peer sees a clean half-close), and the helper completes when both pumps finish, returning (u64, u64) copy counts. Copy errors are EOF-shaped (abrupt close); there is no Err state.

Datagram substrate (udp): mandatory length framing

Every datagram rides [len: u16 BE][datagram] on BOTH directions — mandatory for correctness (the F-2 rationale lives in ADR-003; the short form: raw pass-through cannot carry an empty datagram). Under the codec:

• Framing (frame_datagram): len: u16 BE + payload. Datagrams

  65535 bytes are rejected at frame time (Oversize) — never a wire overflow (the u16 length field would wrap).

• Decoding (DatagramReader): incremental — datagrams split across channel chunks, batch into single chunks, and survive partial headers. The decoder buffers state across reads; it is the only consumer-visible decode path.
• len = 0 is a legal empty datagram (DNS-over-TCP-style zero-payload probes). EOF is exclusively the channels-level sentinel (BiStream EOF after draining); the codec never emits zero-length reads, so the layers never collide. A recv_datagram returns Some(bytes) per datagram (possibly empty), None only on stream EOF.
• Chunk-size independence: datagram boundaries survive chunk splitting, batching, and any channels-layer re-chunking — verified end-to-end at 7-byte chunk splits (forward POC).
• MTU discipline: 1400-byte datagrams (a conservative MTU-safe payload, not the theoretical 1472 max) ride the bounded-buffer path; the 64-parked-chunks bound (alkcall early-arrival cap) is upstream sizing, not tunnel policy.
• Truncation: silent truncation never happens (OQ-TN-13, resolved fail-loud per ADR-003): the local adapter recvs on a full 65535-byte scratch (UDP try_recv truncates silently at the buffer size — a smaller scratch would corrupt the framing) and a

  caller-buffer frame is staged across reads, the length prefix intact, so the far-side de-framer reassembles the true length. A wire-level over-bound datagram (>65535 — impossible for a well-formed peer) is a loud error at the frame path.

Sentinels and lifecycle (both codecs)

• EOF: the channels-level zero-length chunk, written by pump_bidi's shutdown-on-completion or by a channel teardown (wrapper exit, channel/close, transport EOF → demux sender drop). EOF is per-direction (half-close): one direction EOFs, the other keeps pumping until its own EOF.
• No mid-stream control frames. Pump-phase failures are EOF-shaped by design (alkcall ADR-049 §6); there is no tunnel-level error frame, no KEEPALIVE (channels transport liveness is upstream's), no flow-expiry frame (producer-side bookkeeping, and idle-expiry mapping is a producer-registry concern, not wire).
• No establishment frame. The open op's reply IS the establishment result (ADR-049); the pre-ADR-049 "JSON ack before data" hunch is superseded — the wire carries zero tunnel-level control framing in v1.
• BAST: the binary framing (the UDP codec) carries its BAST document — bast.md (AGENTS.md convention 12; the stream pass-through has no binary framing to describe).

Normative byte diagrams

Stream substrate, one direction's wire view (inside the channel BiStream, after the channels layer's 8-byte header per chunk):

<payload bytes> ... <EOF sentinel: channels-level length=0 chunk>

UDP substrate, one direction's wire view:

[len:u16 BE][datagram bytes] [len:u16 BE][datagram bytes] ...
  len == 0: an empty datagram (legal payload, NOT EOF)
  EOF: the channels-level sentinel, outside the codec

Open Questions

• OQ-TN-13: resolved (fail-loud, ADR-003) — see the Truncation bullet
• OQ-TN-02: resolved — see the ADR-003 mandate

References

• ADR-001 (params), ADR-002 (ALPN), ADR-003 (codec), ADR-005 (session teardown)
• Forward POC docs/research/poc-summary.md (codec + sentinel layering, 17 tests); reverse POC docs/research/reverse-poc-summary.md (F-2 mandate, W4 half-close)
• alkcall ADR-047 (open ops), ADR-049 (establishment/typed errors), ADR-050 (pump_bidi); alknet ADR-071/093 (channels wire)