alktunnels/docs/architecture/wire.md

---
status: draft
last_updated: 2026-09-20 (review 002 D-1/D-2 sync: the params schemas' port bounds, `additionalProperties: false` gates, `listen` `minimum: 1`, and the registry-side gate ordering recorded; ADR-007's oneOf wording corrected to the shipped allOf/if-then)
---

# alktunnels: Wire Format

The `alk/tunnel` wire surface. Two layers: the **open op** (JSON
params on the channels call plane: structured, schema-validated,
typed errors) and the **data plane** (bytes/datagrams inside the
channel `BiStream`, the codec, ADR-003). The channels layer strips
its own 8-byte header transparently (alknet ADR-093); the tunnel
protocol owns everything inside the `BiStream`.

Design decisions live in the ADRs ([001](decisions/001-open-params-layout.md),
[002](decisions/002-alpn-strategy.md), [003](decisions/003-codec-and-udp-framing.md),
[007](decisions/007-direct-open-op.md), [008](decisions/008-listen-metadata-forwarded-op.md));
this document is the normative WHAT.

## The Open Op (`channels/tunnel/sub`)

`Sub`-typed operation (alkcall ADR-047): the reply carries
`{ "channel_id": <u32> }` once; the data plane flows on the channel's
`BiStream`, never in the call response stream.

### Input (params)

```json
{
  "resource": "postgres-primary",
  "substrate": "tcp"
}
```

- `resource`: string, required: the producer's stable resource name
  (NOT an address; the producer's registry maps it to its backing).
- `substrate`: string, required, enum `["tcp", "udp", "unix"]`: the
  extensible discriminator (ADR-001); selects the data-plane framing.
  The schema ships with all three values in v1 (unix has an
  implementation path, OQ-TN-14). An older producer rejecting a
  newer substrate value is the SSH "unknown channel type" posture:
  loud, not silent.
- `input_schema` pins both as required strings; unknown substrate
  values fail schema validation (typed error, loud, not silent).
- Extensibility: a new substrate is a new enum value, not a format
  change. An older producer rejecting a newer substrate is the SSH
  "unknown channel type" posture.

### Reply / errors

- Success: `{ "channel_id": <u32> }`: the channel is open, the
  establishment phase (dial) succeeded, the pump handler is running.
  Bind-first listen opens MAY add `"bound"` (§The Listen Open Op's
  Reply below).
- Failure: a typed `channel:open_failed` call error with
  `details: { reason, message }` with `reason ∈ dial_failed` /
  `unknown_resource` / `resource_shortage` / `handler_error` /
  `timeout` (alkcall ADR-049 §3). The registry's ACL gate runs before
  the wrapper (`FORBIDDEN` for a scope-less identity); the 256-channel
  cap is `channel:too_many_channels`. **A failed open never returns a
  `channel_id`**; no phantom channel (the SSH contract, POC-verified
  from the consumer side).
- The scope gate: the open op's `AccessControl.required_scopes` is
  `["tunnel:open"]` (the `TUNNEL_OPEN_SCOPE` constant). Caller
  identity resolves in the alkcall 0.7.0 precedence order: payload
  `auth_token` > `ServingConfig.identity` > transport identity (CF-005);
  the establisher and pump handler receive the per-call opener
  identity (CF-006).

## The Direct Op (`channels/tunnel/direct`)

Dynamic-target egress (ADR-007): "dial what I name," not "dial what
the producer registered." `Sub`-typed, ALPN marker `alk/tunnel`; the
same establishment phase and wrapper as `…/sub`. Scope gate:
`["tunnel:direct"]` (`TUNNEL_DIRECT_SCOPE`), **never implied by
`tunnel:open`** (ADR-006 Amendment 1).

### Input (params)

```json
{
  "substrate": "tcp",
  "target": { "host": "db.internal", "port": 5432 }
}
```

- `substrate`: string, required, enum `["tcp", "udp", "unix"]`: the
  same discriminator semantics as `…/sub` (ADR-001); selects the
  data-plane framing.
- `target`: object, required, keyed by the sibling `substrate`:
  - `tcp`/`udp` → `{ "host": <string>, "port": <u16> }` (domain-form
    hosts stay unresolved on the wire; producer-side resolution, the
    ssh `-D` semantic);
  - `unix` → `{ "path": <string> }`.
- `input_schema` pins both required; the target sub-schema is
  per-substrate: an `allOf` of two `if`/`then` branches keyed on the
  sibling `substrate` value, validated **registry-side** (the schema
  gate runs before the establisher, as the behavior
  `malformed_target_is_schema_class_and_leaves_no_channel` pins; the
  establisher's serde parse re-checks as the backstop). Malformed
  targets are the `invalid_input`-class failure; **`unknown_resource`
  can never fire on the direct op** (no registry lookup).
- Schema bounds (wire-stable; the target sub-schema is part of the
  one-way door, ADR-007 §"Wire-stable"): each branch's target object
  carries `"type": "object"` and `"additionalProperties": false`:
  - `tcp`/`udp` → `{ "host": {"type": "string"}, "port": {"type":
    "integer", "minimum": 0, "maximum": 65535} }`, both required;
  - `unix` → `{ "path": {"type": "string"} }`, required.
  
  The `additionalProperties: false` gate is what makes a cross-shape
  target (a `unix` open carrying `host`) fail the registry's
  `INVALID_INPUT` class *before* the establisher, not a serde-level
  quirk.

### Reply / errors

- Success: `{ "channel_id": <u32> }`: identical semantics to
  `…/sub`; no `bound` field (dial establishers do not bind).
- Failure: the same typed surface as `…/sub` minus `unknown_resource`,
  plus the schema/`invalid_input` class for malformed targets.

## The Listen Open Op's Reply: the `bound` and `listen` Fields

A bind-first listen open (ADR-008 §1, served by `…/sub` with the
bind-first establisher) MAY add optional, additive reply fields:

```json
{ "channel_id": 7, "bound": { "host": "203.0.113.9", "port": 42113 },
  "listen": 918273 }
```

- `bound` is substrate-shaped like the direct op's target
  (`{host, port}` / `{path}`); the REAL OS-chosen address (port `0`
  resolved).
- `listen` is the correlation token (ADR-008 Amendment 1): a u32 the
   establisher minted, unique per open on that producer; the key the
  forwarded op's params carry (§The Forwarded Op below). Riding the
  reply is what makes it relay-transparent: reply fields pass through
  hub re-produce untouched (alkcall ADR-051), and the same minted
  token reappears in every hop's reply.
- Present only when the establisher supplies them; absent for dial
  establishers and accept-first listen opens. Old consumers ignore
  unknown reply fields (plain JSON), which is wire-additive.
- `bound` is informational surface, not addressing: a consumer MUST
  NOT treat `bound` as a dial target (the producer owns the backing,
  ADR-001). It exists for RFC-1928-reply-class fidelity (BND.ADDR)
  and observability.

## The Forwarded Op (`channels/tunnel/forwarded`)

Accept-as-open (ADR-008 §3): when a bind-first listener accepts, the
PRODUCER opens a channel back toward the listen-opener side. The
producer *calls* the op (the opener SERVES it, per the ADR-047 §5
connection-owner rule: the serving side allocates the `channel_id`,
the producer-side initiator adopts). `Sub`-typed, ALPN marker
`alk/tunnel`; the serving side registers establisher-less by base
shape (the data plane starts immediately) or (the recommended shape,
ADR-008 Amendment 2) with the correlation-gate establisher variant
(a stale/missing token then fails the op as a typed
`channel:open_failed` with `details.reason = "unknown_resource"`; the
wire vocabulary is unchanged by the amendment).
Scope gate (opener side): `["tunnel:forwarded"]`
(`TUNNEL_FORWARDED_SCOPE`): ingress, a separate grant from
`tunnel:open`/`tunnel:direct` (ADR-006 Amendment 1).

### Input (params)

```json
{
  "listen": 918273,
  "substrate": "tcp",
  "peer": { "host": "198.51.100.4", "port": 55231 }
}
```

- `listen`: integer > 0, required: the correlation token from the
  listen open's `listen` reply field (ADR-008 Amendment 1; the
  producer-minted token, relay-transparent). The opener correlates
  the accept to its waiting listener session by matching it. The
  schema pins `"minimum": 1` and the serde type pins it as the
  backstop (a `listen: 0` on the wire is a malformed open, rejected
   at parse time, never silently accepted).
- `substrate`: string, required, enum: the listen tunnel's
  substrate; shapes the forwarded channel's data-plane framing.
- `peer`: object, required: the accepted connection's remote address,
  substrate-shaped (`{host, port}` / `{path}`). Reply#2-class
  fidelity's carrier (ask #2).
- Schema bounds (wire-stable, mirroring the direct op's target
  sub-schema): the per-substrate `peer` sub-schema is an `allOf` of
  two `if`/`then` branches keyed on the sibling `substrate`, each
   branch's object carrying `"type": "object"` and
   `"additionalProperties": false`:
  - `tcp`/`udp` → `{ "host": {"type": "string"}, "port": {"type":
    "integer", "minimum": 0, "maximum": 65535} }`, both required;
  - `unix` → `{ "path": {"type": "string"} }`, required.
  
  Malformed peers fail the registry's schema gate before the
  opener-side handler runs.

### Reply / errors

- Success: `{ "channel_id": <u32> }`: the opener-allocated ID the
  producer adopts, then pumps the accepted halves against.
- Failure: a stale/missing correlation token is the typed
  `channel:open_failed` with `details.reason = "unknown_resource"`
  (the establisher variant, ADR-008 Amendment 2, which is the
  recommended serving shape; the miss never births a channel). An opener without
  the scope is `FORBIDDEN`; an old opener that does not serve the op
  rejects loudly (`NOT_FOUND`: the SSH posture). Other open failures
  ride the existing typed vocabulary. Under the base (establisher-
  less) serving shape an in-session miss produces no call error; it
  is the assembly-side post-open decline (the channel reaps at
  birth, `correlation-miss` at `debug`).

## The Data Plane

One data stream per direction on one channel: the channel ID is the
flow key (no conn-id, no sub-demux). Two codecs by substrate
(ADR-003):

### Stream substrates (`tcp`, `unix`): raw pass-through

The halves are the tunnel. Zero tunnel-level framing; the only wire
overhead is the channels 8-byte chunk header. A zero-length read is
genuinely EOF (stream semantics).

Both pumps run `alkcall::channels::pump_bidi(channel, peer_read,
peer_write)`, the two-pump contract (alknet ADR-078, pinned upstream
ADR-050): when one direction's source EOFs, the pump shuts the
OPPOSITE sink down (the EOF sentinel crosses the mux; the peer sees a
clean half-close), and the helper completes when both pumps finish,
returning `(u64, u64)` copy counts. Copy errors are EOF-shaped
(abrupt close); there is no `Err` state.

### Datagram substrate (`udp`): mandatory length framing

Every datagram rides `[len: u16 BE][datagram]` on BOTH directions;
this is mandatory for correctness (the F-2 rationale lives in ADR-003; the
short form: raw pass-through cannot carry an empty datagram). Under
the codec:

- **Framing (`frame_datagram`):** `len: u16 BE` + payload. Datagrams
  > 65535 bytes are rejected at frame time (`Oversize`), never a
  wire overflow (the u16 length field would wrap).
- **Decoding (`DatagramReader`):** incremental; datagrams split
  across channel chunks, batch into single chunks, and survive
  partial headers. The decoder buffers state across reads; it is the
  only consumer-visible decode path.
- **`len = 0` is a legal empty datagram** (DNS-over-TCP-style
  zero-payload probes). EOF is exclusively the channels-level
  sentinel (`BiStream` EOF after draining); the codec never emits
  zero-length reads, so the layers never collide. A `recv_datagram`
  returns `Some(bytes)` per datagram (possibly empty), `None` only on
  stream EOF.
- **Chunk-size independence:** datagram boundaries survive chunk
  splitting, batching, and any channels-layer re-chunking, verified
  end-to-end at 7-byte chunk splits (forward POC).
- **MTU discipline:** 1400-byte datagrams (a conservative MTU-safe
  payload, not the theoretical 1472 max) ride the bounded-buffer path;
  the 64-parked-chunks bound (alkcall early-arrival cap) is upstream
  sizing, not tunnel policy.
- **Truncation:** silent truncation never happens (OQ-TN-13, resolved
  fail-loud per ADR-003): the `local` adapter recvs on a full
  65535-byte scratch (UDP `try_recv` truncates silently at the buffer
  size; a smaller scratch would corrupt the framing) and a
  >caller-buffer frame is staged across reads, the length prefix
  intact, so the far-side de-framer reassembles the true length. A
  wire-level over-bound datagram (>65535, impossible for a
  well-formed peer) is a loud error at the frame path.

### Sentinels and lifecycle (both codecs)

- **EOF:** the channels-level zero-length chunk, written by
  `pump_bidi`'s shutdown-on-completion or by a channel teardown
  (wrapper exit, `channel/close`, transport EOF → demux sender drop).
  EOF is per-direction (half-close): one direction EOFs, the other
  keeps pumping until its own EOF.
- **No mid-stream control frames.** Pump-phase failures are
  EOF-shaped by design (alkcall ADR-049 §6); there is no tunnel-level
  error frame, no KEEPALIVE (channels transport liveness is
  upstream's), no flow-expiry frame (producer-side bookkeeping, and
  idle-expiry mapping is a producer-registry concern, not wire).
- **No establishment frame.** The open op's reply IS the
  establishment result (ADR-049); the pre-ADR-049
  "JSON ack before data" hunch is superseded; the wire carries zero
  tunnel-level control framing in v1.
- **BAST:** the binary framing (the UDP codec) carries its BAST
  document in [bast.md](bast.md) (AGENTS.md convention 12; the
  stream pass-through has no binary framing to describe).

## Normative byte diagrams

Stream substrate, one direction's wire view (inside the channel
`BiStream`, after the channels layer's 8-byte header per chunk):

```
<payload bytes> ... <EOF sentinel: channels-level length=0 chunk>
```

UDP substrate, one direction's wire view:

```
[len:u16 BE][datagram bytes] [len:u16 BE][datagram bytes] ...
  len == 0: an empty datagram (legal payload, NOT EOF)
  EOF: the channels-level sentinel, outside the codec
```

## Open Questions

- **OQ-TN-13**: resolved (fail-loud, ADR-003); see the Truncation
  bullet
- **OQ-TN-02**: resolved; see the ADR-003 mandate
- **OQ-TN-15**: resolved; see ADR-007 (the direct op)
- **OQ-TN-16**: resolved; see ADR-008 (the `bound`/`listen` fields +
  the forwarded op)

## References

- [ADR-001](decisions/001-open-params-layout.md) (params, per-op
  amendment),
  [ADR-002](decisions/002-alpn-strategy.md) (ALPN + flavor-form op
  ids),
  [ADR-003](decisions/003-codec-and-udp-framing.md) (codec),
  [ADR-005](decisions/005-consumer-session-owns-teardown.md) (session
  teardown),
  [ADR-007](decisions/007-direct-open-op.md) (the direct op),
  [ADR-008](decisions/008-listen-metadata-forwarded-op.md) (listen
  metadata + the forwarded op)
- Forward POC `docs/research/poc-summary.md` (codec + sentinel
  layering, 17 tests); reverse POC
  `docs/research/reverse-poc-summary.md` (F-2 mandate, W4 half-close)
- alkcall ADR-047 (open ops), ADR-049 (establishment/typed errors),
  ADR-050 (`pump_bidi`); alknet ADR-071/093 (channels wire)
- `docs/research/tunnels-graduation.md` (the graduation research the
  new ops implement); the upstream asks (review 008) landed in
  alkcall 0.8.0; see
  `alkcall/docs/reviews/008-tunnels-graduation-upstream-review.md`