alktunnels/docs/architecture/wire.md

---

status: draft last_updated: 2026-09-20 (review 002 D-1/D-2 sync: the params schemas' port bounds, additionalProperties: false gates, listen minimum: 1, and the registry-side gate ordering recorded; ADR-007's oneOf wording corrected to the shipped allOf/if-then)

alktunnels — Wire Format

The alk/tunnel wire surface. Two layers: the open op (JSON params on the channels call plane — structured, schema-validated, typed errors) and the data plane (bytes/datagrams inside the channel BiStream — the codec, ADR-003). The channels layer strips its own 8-byte header transparently (alknet ADR-093); the tunnel protocol owns everything inside the BiStream.

Design decisions live in the ADRs (001, 002, 003, 007, 008); this document is the normative WHAT.

The Open Op (channels/tunnel/sub)

Sub-typed operation (alkcall ADR-047): the reply carries { "channel_id": <u32> } once; the data plane flows on the channel's BiStream — never in the call response stream.

Input (params)

{
  "resource": "postgres-primary",
  "substrate": "tcp"
}

• resource: string, required — the producer's stable resource name (NOT an address; the producer's registry maps it to its backing).
• substrate: string, required, enum ["tcp", "udp", "unix"] — the extensible discriminator (ADR-001); selects the data-plane framing. The schema ships with all three values in v1 (unix has an implementation path — OQ-TN-14). An older producer rejecting a newer substrate value is the SSH "unknown channel type" posture — loud, not silent.
• input_schema pins both as required strings; unknown substrate values fail schema validation (typed error — loud, not silent).
• Extensibility: a new substrate is a new enum value, not a format change. An older producer rejecting a newer substrate is the SSH "unknown channel type" posture.

Reply / errors

• Success: { "channel_id": <u32> } — the channel is open, the establishment phase (dial) succeeded, the pump handler is running. Bind-first listen opens MAY add "bound" (§The Listen Open Op's Reply below).
• Failure: a typed channel:open_failed call error with details: { reason, message } — reason ∈ dial_failed / unknown_resource / resource_shortage / handler_error / timeout (alkcall ADR-049 §3). The registry's ACL gate runs before the wrapper (FORBIDDEN for a scope-less identity); the 256-channel cap is channel:too_many_channels. A failed open never returns a channel_id — no phantom channel (the SSH contract, POC-verified from the consumer side).
• The scope gate: the open op's AccessControl.required_scopes is ["tunnel:open"] (the TUNNEL_OPEN_SCOPE constant). Caller identity resolves in the alkcall 0.7.0 precedence order: payload auth_token > ServingConfig.identity > transport identity (CF-005); the establisher and pump handler receive the per-call opener identity (CF-006).

The Direct Op (channels/tunnel/direct)

Dynamic-target egress (ADR-007) — "dial what I name," not "dial what the producer registered." Sub-typed, ALPN marker alk/tunnel; the same establishment phase and wrapper as …/sub. Scope gate: ["tunnel:direct"] (TUNNEL_DIRECT_SCOPE) — never implied by tunnel:open (ADR-006 Amendment 1).

Input (params)

{
  "substrate": "tcp",
  "target": { "host": "db.internal", "port": 5432 }
}

• substrate: string, required, enum ["tcp", "udp", "unix"] — the same discriminator semantics as …/sub (ADR-001); selects the data-plane framing.

• target: object, required, keyed by the sibling substrate —

  • tcp/udp → { "host": <string>, "port": <u16> } (domain-form hosts stay unresolved on the wire — producer-side resolution, the ssh -D semantic);
  • unix → { "path": <string> }.

• input_schema pins both required; the target sub-schema is per-substrate — an allOf of two if/then branches keyed on the sibling substrate value, validated registry-side (the schema gate runs before the establisher — the behavior malformed_target_is_schema_class_and_leaves_no_channel pins; the establisher's serde parse re-checks as the backstop). Malformed targets are the invalid_input-class failure; unknown_resource can never fire on the direct op (no registry lookup).

• Schema bounds (wire-stable — the target sub-schema is part of the one-way door, ADR-007 §"Wire-stable"): each branch's target object carries "type": "object" and "additionalProperties": false —

  • tcp/udp → { "host": {"type": "string"}, "port": {"type": "integer", "minimum": 0, "maximum": 65535} }, both required;
  • unix → { "path": {"type": "string"} }, required.

  The additionalProperties: false gate is what makes a cross-shape target (a unix open carrying host) fail the registry's INVALID_INPUT class before the establisher — not a serde-level quirk.

Reply / errors

• Success: { "channel_id": <u32> } — identical semantics to …/sub; no bound field (dial establishers do not bind).
• Failure: the same typed surface as …/sub minus unknown_resource, plus the schema/invalid_input class for malformed targets.

The Listen Open Op's Reply — the bound and listen Fields

A bind-first listen open (ADR-008 §1, served by …/sub with the bind-first establisher) MAY add optional, additive reply fields:

{ "channel_id": 7, "bound": { "host": "203.0.113.9", "port": 42113 },
  "listen": 918273 }

• bound is substrate-shaped like the direct op's target ({host, port} / {path}); the REAL OS-chosen address (port 0 resolved).
• listen is the correlation token (ADR-008 Amendment 1): a u32 the establisher minted, unique per open on that producer — the key the forwarded op's params carry (§The Forwarded Op below). Riding the reply is what makes it relay-transparent: reply fields pass through hub re-produce untouched (alkcall ADR-051), and the same minted token reappears in every hop's reply.
• Present only when the establisher supplies them; absent for dial establishers and accept-first listen opens. Old consumers ignore unknown reply fields (plain JSON) — wire-additive.
• bound is informational surface, not addressing: a consumer MUST NOT treat bound as a dial target (the producer owns the backing — ADR-001). It exists for RFC-1928-reply-class fidelity (BND.ADDR) and observability.

The Forwarded Op (channels/tunnel/forwarded)

Accept-as-open (ADR-008 §3): when a bind-first listener accepts, the PRODUCER opens a channel back toward the listen-opener side. The producer calls the op (the opener SERVES it — the ADR-047 §5 connection-owner rule: the serving side allocates the channel_id, the producer-side initiator adopts). Sub-typed, ALPN marker alk/tunnel; no establisher (the data plane starts immediately). Scope gate (opener side): ["tunnel:forwarded"] (TUNNEL_FORWARDED_SCOPE) — ingress, a separate grant from tunnel:open/tunnel:direct (ADR-006 Amendment 1).

Input (params)

{
  "listen": 918273,
  "substrate": "tcp",
  "peer": { "host": "198.51.100.4", "port": 55231 }
}

• listen: integer > 0, required — the correlation token from the listen open's listen reply field (ADR-008 Amendment 1; the producer-minted token, relay-transparent). The opener correlates the accept to its waiting listener session by matching it. The schema pins "minimum": 1 and the serde type pins it as the backstop (a listen: 0 on the wire is a malformed open, rejected at parse time — never silently accepted).

• substrate: string, required, enum — the listen tunnel's substrate; shapes the forwarded channel's data-plane framing.

• peer: object, required — the accepted connection's remote address, substrate-shaped ({host, port} / {path}). Reply#2-class fidelity's carrier (ask #2).

• Schema bounds (wire-stable, mirroring the direct op's target sub-schema): the per-substrate peer sub-schema is an allOf of two if/then branches keyed on the sibling substrate, each branch's object carrying "type": "object" and "additionalProperties": false —

  • tcp/udp → { "host": {"type": "string"}, "port": {"type": "integer", "minimum": 0, "maximum": 65535} }, both required;
  • unix → { "path": {"type": "string"} }, required.

  Malformed peers fail the registry's schema gate before the opener-side handler runs.

Reply / errors

• Success: { "channel_id": <u32> } — the opener-allocated ID the producer adopts, then pumps the accepted halves against.
• Failure: a producer-side accept correlation miss is the typed channel:open_failed/handler_error-class error; an opener without the scope is FORBIDDEN; an old opener that does not serve the op rejects loudly (NOT_FOUND — the SSH posture).

The Data Plane

One data stream per direction on one channel — the channel ID is the flow key (no conn-id, no sub-demux). Two codecs by substrate (ADR-003):

Stream substrates (tcp, unix): raw pass-through

The halves are the tunnel. Zero tunnel-level framing; the only wire overhead is the channels 8-byte chunk header. A zero-length read is genuinely EOF (stream semantics).

Both pumps run alkcall::channels::pump_bidi(channel, peer_read, peer_write) — the two-pump contract (alknet ADR-078, pinned upstream ADR-050): when one direction's source EOFs, the pump shuts the OPPOSITE sink down (the EOF sentinel crosses the mux; the peer sees a clean half-close), and the helper completes when both pumps finish, returning (u64, u64) copy counts. Copy errors are EOF-shaped (abrupt close); there is no Err state.

Datagram substrate (udp): mandatory length framing

Every datagram rides [len: u16 BE][datagram] on BOTH directions — mandatory for correctness (the F-2 rationale lives in ADR-003; the short form: raw pass-through cannot carry an empty datagram). Under the codec:

• Framing (frame_datagram): len: u16 BE + payload. Datagrams

  65535 bytes are rejected at frame time (Oversize) — never a wire overflow (the u16 length field would wrap).

• Decoding (DatagramReader): incremental — datagrams split across channel chunks, batch into single chunks, and survive partial headers. The decoder buffers state across reads; it is the only consumer-visible decode path.
• len = 0 is a legal empty datagram (DNS-over-TCP-style zero-payload probes). EOF is exclusively the channels-level sentinel (BiStream EOF after draining); the codec never emits zero-length reads, so the layers never collide. A recv_datagram returns Some(bytes) per datagram (possibly empty), None only on stream EOF.
• Chunk-size independence: datagram boundaries survive chunk splitting, batching, and any channels-layer re-chunking — verified end-to-end at 7-byte chunk splits (forward POC).
• MTU discipline: 1400-byte datagrams (a conservative MTU-safe payload, not the theoretical 1472 max) ride the bounded-buffer path; the 64-parked-chunks bound (alkcall early-arrival cap) is upstream sizing, not tunnel policy.
• Truncation: silent truncation never happens (OQ-TN-13, resolved fail-loud per ADR-003): the local adapter recvs on a full 65535-byte scratch (UDP try_recv truncates silently at the buffer size — a smaller scratch would corrupt the framing) and a

  caller-buffer frame is staged across reads, the length prefix intact, so the far-side de-framer reassembles the true length. A wire-level over-bound datagram (>65535 — impossible for a well-formed peer) is a loud error at the frame path.

Sentinels and lifecycle (both codecs)

• EOF: the channels-level zero-length chunk, written by pump_bidi's shutdown-on-completion or by a channel teardown (wrapper exit, channel/close, transport EOF → demux sender drop). EOF is per-direction (half-close): one direction EOFs, the other keeps pumping until its own EOF.
• No mid-stream control frames. Pump-phase failures are EOF-shaped by design (alkcall ADR-049 §6); there is no tunnel-level error frame, no KEEPALIVE (channels transport liveness is upstream's), no flow-expiry frame (producer-side bookkeeping, and idle-expiry mapping is a producer-registry concern, not wire).
• No establishment frame. The open op's reply IS the establishment result (ADR-049); the pre-ADR-049 "JSON ack before data" hunch is superseded — the wire carries zero tunnel-level control framing in v1.
• BAST: the binary framing (the UDP codec) carries its BAST document — bast.md (AGENTS.md convention 12; the stream pass-through has no binary framing to describe).

Normative byte diagrams

Stream substrate, one direction's wire view (inside the channel BiStream, after the channels layer's 8-byte header per chunk):

<payload bytes> ... <EOF sentinel: channels-level length=0 chunk>

UDP substrate, one direction's wire view:

[len:u16 BE][datagram bytes] [len:u16 BE][datagram bytes] ...
  len == 0: an empty datagram (legal payload, NOT EOF)
  EOF: the channels-level sentinel, outside the codec

Open Questions

• OQ-TN-13: resolved (fail-loud, ADR-003) — see the Truncation bullet
• OQ-TN-02: resolved — see the ADR-003 mandate
• OQ-TN-15: resolved — see ADR-007 (the direct op)
• OQ-TN-16: resolved — see ADR-008 (the bound/listen fields + the forwarded op)

References

• ADR-001 (params, per-op amendment), ADR-002 (ALPN + flavor-form op ids), ADR-003 (codec), ADR-005 (session teardown), ADR-007 (the direct op), ADR-008 (listen metadata + the forwarded op)
• Forward POC docs/research/poc-summary.md (codec + sentinel layering, 17 tests); reverse POC docs/research/reverse-poc-summary.md (F-2 mandate, W4 half-close)
• alkcall ADR-047 (open ops), ADR-049 (establishment/typed errors), ADR-050 (pump_bidi); alknet ADR-071/093 (channels wire)
• docs/research/tunnels-graduation.md (the graduation research the new ops implement); the upstream asks (review 008) landed in alkcall 0.8.0 — see alkcall/docs/reviews/008-tunnels-graduation-upstream-review.md