diff --git a/docs/architecture/README.md b/docs/architecture/README.md index e792078..1671584 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -3,13 +3,13 @@ status: draft last_updated: 2026-09-16 (the graduation ADRs 007/008 + the amendments) --- -# alktunnels — Architecture +# alktunnels: Architecture Arbitrary bidirectional tunnels over alkcall channels: the `alk/tunnel`-family producer/consumer protocol crate. This directory holds the architecture spec docs and the ADRs. It is the sibling of -alktty's architecture (`/workspace/@alkdev/alktty/docs/architecture/`) -— same substrate (alkcall channels), same shape (a producer +alktty's architecture (`/workspace/@alkdev/alktty/docs/architecture/`): +same substrate (alkcall channels), same shape (a producer open-handler + a typed consumer session), different service (tunnels: raw byte/datagram transport, not terminal sessions). @@ -18,41 +18,41 @@ raw byte/datagram transport, not terminal sessions). | Document | Status | Description | |----------|--------|-------------| | [overview.md](overview.md) | draft | Crate purpose, the resource model in brief, dependencies, ALPN, feature gates, module map | -| [wire.md](wire.md) | draft | The wire format: open-op params JSON (ADR-001; the direct op's `{substrate, target}` — ADR-007), the `bound` reply field + the forwarded op (ADR-008), the data-plane codec (raw pass-through for stream substrates; mandatory `[len: u16 BE]` framing for UDP — ADR-003), sentinels, half-close semantics | +| [wire.md](wire.md) | draft | The wire format: open-op params JSON (ADR-001; the direct op's `{substrate, target}` per ADR-007), the `bound` reply field + the forwarded op (ADR-008), the data-plane codec (raw pass-through for stream substrates; mandatory `[len: u16 BE]` framing for UDP per ADR-003), sentinels, half-close semantics | | [producer.md](producer.md) | draft | The producer half: the `channels/tunnel/sub` + `channels/tunnel/direct` open ops, three establisher shapes (dial, accept-first listen, bind-first listen), the pump handler, the forwarded-op opener side, registration, scope gating | | [consumer.md](consumer.md) | draft | The consumer half: `TunnelSession` (stream + datagram variants, `open`/`open_direct`), `TunnelListener` (the listen flow), the reverse-flow initiator, teardown ownership (ADR-005/008) | -| [bast.md](bast.md) | draft | The BAST (Binary Abstract Syntax Tree) document for the data-plane's binary framing (the UDP `[len: u16 BE]` codec — ADR-003); the stream pass-through has no binary framing to describe | +| [bast.md](bast.md) | draft | The BAST (Binary Abstract Syntax Tree) document for the data-plane's binary framing (the UDP `[len: u16 BE]` codec, ADR-003); the stream pass-through has no binary framing to describe | | [open-questions.md](open-questions.md) | draft | Centralized OQ tracker (OQ-TN-01..16; the Phase 0 ledger promoted) | ## Applicable ADRs alktunnels-native ADRs (001..008). Upstream ADRs (alkcall/alknet) are -referenced by their own numbers and are NOT ported — the alkcall +referenced by their own numbers and are NOT ported; the alkcall originals at `/workspace/@alkdev/alkcall/docs/architecture/decisions/` remain authoritative for the shared surfaces (establishment, channels wire format, `pump_bidi`, identity seam). | ADR | Title | Status | |-----|-------|--------| -| [001](decisions/001-open-params-layout.md) | Open-Op Params Layout: Resource + Substrate | Accepted; amended (params layout is per-op — the direct op's params ride ADR-007) | +| [001](decisions/001-open-params-layout.md) | Open-Op Params Layout: Resource + Substrate | Accepted; amended (params layout is per-op; the direct op's params ride ADR-007) | | [002](decisions/002-alpn-strategy.md) | Single `alk/tunnel` ALPN (Option A) | Accepted; amended (flavor-form op ids: `…/sub`, `…/direct`, `…/forwarded` on the one ALPN) | | [003](decisions/003-codec-and-udp-framing.md) | Data-Plane Codec: Raw Pass-Through (Stream) / Mandatory Length Framing (UDP) | Accepted | -| [004](decisions/004-no-backend-trait.md) | No TunnelBackend Trait — Halves Functions at the Assembly Layer | Accepted | +| [004](decisions/004-no-backend-trait.md) | No TunnelBackend Trait: Halves Functions at the Assembly Layer | Accepted | | [005](decisions/005-consumer-session-owns-teardown.md) | The Consumer Session Type Owns Teardown | Accepted | -| [006](decisions/006-access-control-posture.md) | Access-Control Posture — the Open Gate Is the Security Boundary | Accepted; amended (`tunnel:direct`/`tunnel:forwarded` scopes; the hub re-produce policy sentence) | -| [007](decisions/007-direct-open-op.md) | The Direct Open Op — Dynamic-Target Egress | Accepted | -| [008](decisions/008-listen-metadata-forwarded-op.md) | Listen Metadata — Bind-First Establisher and the Forwarded Op | Accepted | +| [006](decisions/006-access-control-posture.md) | Access-Control Posture: the Open Gate Is the Security Boundary | Accepted; amended (`tunnel:direct`/`tunnel:forwarded` scopes; the hub re-produce policy sentence) | +| [007](decisions/007-direct-open-op.md) | The Direct Open Op: Dynamic-Target Egress | Accepted | +| [008](decisions/008-listen-metadata-forwarded-op.md) | Listen Metadata: Bind-First Establisher and the Forwarded Op | Accepted | ## Key Design Principles -1. **A tunnel is a resource, not an address — with one additive - exception.** Base params identify a produced resource + substrate +1. **A tunnel is a resource, not an address (with one additive + exception).** Base params identify a produced resource + substrate discriminator; the producer owns the backing (local port, docker container, in-process service, unix socket). Rich addressing enters through `-D`/dynamic composition, which speaks its protocol inside the tunnel payload. The exception (ADR-007): the direct op - lets a consumer ask the producer to dial an arbitrary target — a - different capability on its own op and scope (`tunnel:direct`), + lets a consumer ask the producer to dial an arbitrary target. That + is a different capability on its own op and scope (`tunnel:direct`), never implied by `tunnel:open`. See [ADR-001](decisions/001-open-params-layout.md), [ADR-007](decisions/007-direct-open-op.md), and @@ -61,7 +61,7 @@ wire format, `pump_bidi`, identity seam). 2. **Role follows the resource; connection direction is independent.** Whoever can reach the target is the producer (registers openable channels); whoever wants the bytes is the consumer. `-L` and `-R` - are the same open op with the entry point on different machines — + are the same open op with the entry point on different machines; no protocol-level direction (the hub-owns-the-connection model, OQ-TN-03). The composed `-R` flow rides the bind-first establisher + the forwarded op (ADR-008). Validated by the reverse-flow POC: @@ -70,15 +70,15 @@ wire format, `pump_bidi`, identity seam). [consumer.md](consumer.md). 3. **Raw pass-through for stream substrates; mandatory length framing - for UDP.** A tunnel has one data stream per direction — no + for UDP.** A tunnel has one data stream per direction: no sub-demux key, no tunnel-level header on stream substrates (0 B tunnel overhead; the channels 8-byte header is the only cost). - UDP rides `[len: u16 BE]` framing — mandatory for correctness + UDP rides `[len: u16 BE]` framing, which is mandatory for correctness (F-2; the rationale and the executable pin live in ADR-003). See [ADR-003](decisions/003-codec-and-udp-framing.md) and [wire.md](wire.md). -4. **The two-pump shape is `alkcall::channels::pump_bidi` — never +4. **The two-pump shape is `alkcall::channels::pump_bidi`; it is never hand-rolled.** The producer's pump handler and the consumer-side pump both consume the upstream helper (ADR-050); each pump shuts down the opposite sink on completion (half-close semantics fall @@ -87,7 +87,7 @@ wire format, `pump_bidi`, identity seam). [producer.md](producer.md). 5. **No backend trait.** Stream substrates need "produce boxed halves - for a resource" — a function, not a trait (the UDP POC's OQ-TN-05 + for a resource"; that is a function, not a trait (the UDP POC's OQ-TN-05 finding). The inversion point is the assembly layer wiring substrate halves into the establisher/pump; substrate types stay in feature-gated backend modules. The listen variant (producer @@ -96,7 +96,7 @@ wire format, `pump_bidi`, identity seam). 6. **The consumer session type owns teardown.** One session = one channel = one tunnel (the channel ID is the flow key). The adopted - side has no wrapper awaiting its pump (reverse POC W3) — the + side has no wrapper awaiting its pump (reverse POC W3): the session must hold the pump handle and reap the channel on completion/close. See [ADR-005](decisions/005-consumer-session-owns-teardown.md) and [consumer.md](consumer.md). @@ -110,13 +110,13 @@ wire format, `pump_bidi`, identity seam). 8. **Security posture: the open gate is the boundary.** Scope-gated opens (`TUNNEL_OPEN_SCOPE = "tunnel:open"`, `TUNNEL_DIRECT_SCOPE = "tunnel:direct"`, - `TUNNEL_FORWARDED_SCOPE = "tunnel:forwarded"` — each stable once + `TUNNEL_FORWARDED_SCOPE = "tunnel:forwarded"` (each stable once published; ADR-006 + Amendment 1), alkcall's op-level ACL via `register_openable`; caller identity resolves in the 0.7.0 precedence order (token > `ServingConfig.identity` > transport); the establisher/pump handler see the per-call opener identity. No target allowlists in the protocol (an assembly establisher MAY - enforce them — policy, not protocol). Hub re-produce: the hub's + enforce them, but that is policy, not protocol). Hub re-produce: the hub's ACL governs the final dial (the hub is the network boundary). See [ADR-006](decisions/006-access-control-posture.md), [ADR-007](decisions/007-direct-open-op.md), and @@ -155,10 +155,10 @@ Questions section. (CF-005/CF-006 = the identity seam, resolved in 0.7.0) - The graduation research (the two new ADRs implement it): `docs/research/tunnels-graduation.md`; the upstream alkcall asks - (review 008) **landed in alkcall 0.8.0** — flavor-form discovery + (review 008) **landed in alkcall 0.8.0**: flavor-form discovery derivation + the establisher → reply `bound` projection, plus the in-tree `ChannelRelay`/`HubLegTemplate` (alkcall ADR-051) -- Sibling precedent: alktty (`/workspace/@alkdev/alktty`) — the +- Sibling precedent: alktty (`/workspace/@alkdev/alktty`), i.e. the producer/consumer-on-channels shape, wasm-clean default, feature-gated local backend, BAST convention - POC code: `/workspace/alktunnels-udp-poc` (forward), diff --git a/docs/architecture/bast.md b/docs/architecture/bast.md index e63c894..8213907 100644 --- a/docs/architecture/bast.md +++ b/docs/architecture/bast.md @@ -3,11 +3,11 @@ status: implemented last_updated: 2026-09-10 --- -# alktunnels — BAST Document for the `alk/tunnel` Wire Format +# alktunnels: BAST Document for the `alk/tunnel` Wire Format This document is the **BAST (Binary Abstract Syntax Tree)** -specification for the binary portions of the `alk/tunnel` wire format -— per AGENTS.md convention 12, the mandatory UDP length-framing codec +specification for the binary portions of the `alk/tunnel` wire format. +Per AGENTS.md convention 12, the mandatory UDP length-framing codec ([ADR-003](decisions/003-codec-and-udp-framing.md)) is binary framing beyond pass-through, so the BAST document exists even though the format is trivial. It is a normative JSON document conforming to the @@ -24,31 +24,31 @@ an optional dep and this document is there to feed it. ## Scope The `alk/tunnel` wire format is: a JSON open op on the channels call -plane (a UTF-8 JSON object inside the channels envelope — not a +plane (a UTF-8 JSON object inside the channels envelope, not a binary layout; specified in [wire.md](wire.md) §The Open Op and implemented by `TunnelParams`/`tunnel_open_spec` in the Rust source), and a binary data plane inside the channel `BiStream`. Per the BAST format spec, a BAST document describes **binary data -layouts** — not JSON payloads +layouts**, not JSON payloads ([`validate_bytes` vs `validate_json`](https://alk.dev/bast/v1/spec)). This document therefore covers only the data plane's binary framing: -1. **The datagram frame** (`[len: u16 BE][datagram bytes]`) — the +1. **The datagram frame** (`[len: u16 BE][datagram bytes]`), which is the UDP-substrate codec (ADR-003; mandatory, F-2). Stream substrates - ride raw pass-through — zero tunnel-level framing, so there is + ride raw pass-through, i.e. zero tunnel-level framing, so there is nothing for BAST to describe there. What this BAST deliberately does **not** cover: -- **The open-op params JSON shape** (`{resource, substrate}`) — UTF-8 +- **The open-op params JSON shape** (`{resource, substrate}`), which is UTF-8 JSON on the call plane; specified in [ADR-001](decisions/001-open-params-layout.md) and [wire.md](wire.md), implemented by `TunnelParams`. -- **The channels 8-byte chunk header** — the channels layer's own +- **The channels 8-byte chunk header**, the channels layer's own framing (alknet ADR-071/093); stripped transparently before the tunnel data plane sees bytes. Specified upstream, not here. -- **The EOF sentinel** — the channels-level zero-length chunk +- **The EOF sentinel**, i.e. the channels-level zero-length chunk (upstream); the tunnel codec never emits it (that is the F-2 invariant). @@ -72,12 +72,12 @@ What this BAST deliberately does **not** cover: ## Annotations -### `DatagramFrame` — binary, exact +### `DatagramFrame`: binary, exact Maps 1:1 to the on-wire bytes. `len` as `uint16` big-endian (2 bytes) followed by exactly `len` bytes of datagram payload. The zero-byte -payload (`len == 0`) is a legal empty datagram — a real datagram, NOT -EOF (the F-2 invariant; EOF is the channels-level sentinel, outside +payload (`len == 0`) is a legal empty datagram (a real datagram, NOT +EOF; the F-2 invariant; EOF is the channels-level sentinel, outside this codec). The `maxLength: 65535` encodes the u16 bound; the codec rejects framing a larger datagram at frame time (`Oversize`) rather than wrapping the length field. @@ -85,13 +85,13 @@ than wrapping the length field. One frame per datagram, in both directions of the channel `BiStream`. Frames are self-delimiting on the byte stream; a decoder is an incremental state machine across chunk boundaries (chunk -splitting/batching is transparent — forward POC, 7-byte chunk splits +splitting/batching is transparent, per the forward POC's 7-byte chunk splits verified). ### Layering note The codec sits at the substrate/pump boundary (the establisher wraps -the dialed UDP half in the framed adapter — ADR-003's placement, so +the dialed UDP half in the framed adapter, per ADR-003's placement, so the pump handler stays substrate-agnostic). The BAST describes the wire inside the `BiStream` only; the pump/pump_bidi contract (ADR-050) is upstream's, not re-specified here. @@ -100,9 +100,9 @@ wire inside the `BiStream` only; the pump/pump_bidi contract - [ADR-003](decisions/003-codec-and-udp-framing.md) (the codec decision + F-2 mandate), [ADR-001](decisions/001-open-params-layout.md) - (the JSON open-op shape — outside BAST scope) + (the JSON open-op shape, outside BAST scope) - [wire.md](wire.md) (the normative prose; §Normative byte diagrams) -- alktty `tty-bast.md` (the BAST precedent — same meta-schema, same +- alktty `tty-bast.md` (the BAST precedent, with the same meta-schema and the same "binary framing only" scope rule) - Forward POC `docs/research/poc-summary.md` §3 (the codec's validation matrix); reverse POC diff --git a/docs/architecture/consumer.md b/docs/architecture/consumer.md index 35b889c..29a3886 100644 --- a/docs/architecture/consumer.md +++ b/docs/architecture/consumer.md @@ -1,16 +1,16 @@ --- status: draft -last_updated: 2026-09-18 (`open_direct`, `TunnelListener`, the forwarded-op serving side — ADR-007/008 as amended: the correlation token) +last_updated: 2026-09-18 (`open_direct`, `TunnelListener`, the forwarded-op serving side; ADR-007/008 as amended: the correlation token) --- -# alktunnels — Consumer Half +# alktunnels: Consumer Half -The consumer half: `TunnelSession` — the typed client for tunnel -channels — plus `TunnelListener` (the listen-flow session, ADR-008) +The consumer half: `TunnelSession`, the typed client for tunnel +channels, plus `TunnelListener` (the listen-flow session, ADR-008) and the forwarded-op serving side. The consumer is whoever wants the bytes; it opens tunnel channels toward the producer (forward path), or per local accept toward a connect-side serving producer (the reverse -flow). Both are the same session type — role follows the resource, not +flow). Both are the same session type; role follows the resource, not the connection (OQ-TN-03). Design decisions: [ADR-005](decisions/005-consumer-session-owns-teardown.md) (session owns teardown), [ADR-003](decisions/003-codec-and-udp-framing.md) @@ -20,24 +20,24 @@ metadata + the forwarded op); this document is the normative WHAT. ## Opening a Tunnel -### Forward path (`-L`/direct — the consumer dials the producer's transport) +### Forward path (`-L`/direct, the consumer dials the producer's transport) ```rust let session = TunnelSession::open(&channel_client, params).await?; ``` -- `params`: `{resource, substrate}` (ADR-001) — the produced resource +- `params`: `{resource, substrate}` (ADR-001), i.e. the produced resource + substrate discriminator. - Flow: `ChannelClient::open_channel("channels/tunnel/sub", params, "alk/tunnel")` → adopt the returned channel ID → split the channel `BiStream` → present the substrate-shaped data plane. -- Errors: a failed open resolves a typed error — +- Errors: a failed open resolves a typed error: `ChannelOpenError::CallFailed` carrying the wire `CallError`; branch on `establishment_reason()` for `channel:open_failed`'s reason (dial_failed / unknown_resource / resource_shortage / handler_error / timeout), or the pre-establishment codes (`FORBIDDEN`, `channel:too_many_channels`). **A failed open never - yields a session** — no phantom session, mirroring the + yields a session**; no phantom session, mirroring the no-phantom-channel property (POC-verified). ### Direct path (dynamic-target egress, ADR-007) @@ -51,20 +51,20 @@ let session = TunnelSession::open_direct(&client, target, substrate).await?; caller states which one it holds. The session type, data planes, and teardown API are identical. - `target`: the substrate-shaped JSON object (`{host, port}` / - `{path}` — ADR-007 §2); domain-form hosts stay unresolved (the + `{path}`; ADR-007 §2); domain-form hosts stay unresolved (the producer resolves). - Flow: `open_channel("channels/tunnel/direct", {substrate, target}, "alk/tunnel")` → same adopt/split/data-plane flow as `open`. - Errors: the same typed surface minus `unknown_resource` (no registry lookup); malformed targets are the schema/ `invalid_input`-class failure. An old producer rejects the unknown - op loudly (`NOT_FOUND` — the SSH posture). + op loudly (`NOT_FOUND`, the SSH posture). -### Reverse path (`-R` — the consumer initiates toward a connect-side serving producer) +### Reverse path (`-R`, the consumer initiates toward a connect-side serving producer) The hub/accept-side shape (the reverse POC's `ReverseTunnel`, the seed of this path). Construction differs from the forward path only -in how the channel is obtained — the session type and teardown API +in how the channel is obtained; the session type and teardown API are identical: ```rust @@ -74,29 +74,29 @@ let session = TunnelSession::adopt(hub_manager, channel_id, substrate, TUNNEL_AL let pumped = session.pump_against(accepted_halves).await; // or drive the halves directly ``` -- `open_reverse_channel` — call the worker's open op on channel 0 +- `open_reverse_channel`: call the worker's open op on channel 0 (`call_with_payload`; optional `auth_token`), extract the worker-allocated `channel_id`. Caller identity rides the transport by default (alkcall 0.7.0 CF-005 (b): `Connection::set_identity` before dialing); the payload `auth_token` is the optional hub-forwarding path (precedence: token > `ServingConfig.identity` > transport). -- `TunnelSession::adopt(manager, channel_id, substrate, alpn)` — - adopt the worker-allocated ID (connection-owner rule, ADR-047 §5 — +- `TunnelSession::adopt(manager, channel_id, substrate, alpn)`: + adopt the worker-allocated ID (connection-owner rule, ADR-047 §5: the SERVING side allocates; the initiator adopts; early-arrival parking covers the adoption race), install the session's data plane - from the adopted halves. The substrate parameter is required — a + from the adopted halves. The substrate parameter is required; a datagram session cannot exist without it (the data plane variant is substrate-shaped). `adopt` takes the same typed-error surface as `open` (an adopt failure is `AdoptFailed`-class, not an establishment error). -- `pump_against(accepted_halves)` — spawn `pump_bidi` against the +- `pump_against(accepted_halves)`: spawn `pump_bidi` against the accepted local connection and hold the pump handle (ADR-005). -The session type and its teardown API are identical on both paths — +The session type and its teardown API are identical on both paths: one API, `-L` and `-R` alike. -## The Listen Flow — `TunnelListener` (ADR-008) +## The Listen Flow: `TunnelListener` (ADR-008) The listen-flow sibling of `TunnelSession`: the session that OPENS a bind-first listen tunnel and owns its teardown (ADR-005's ownership @@ -106,25 +106,25 @@ opener's manager (`adopt` gains nothing). ```rust let listener = TunnelListener::open(&client, params).await?; // bind-first: fast establishment -let bound = listener.bound(); // the resolved OS-chosen address (Option — absent when unknown) +let bound = listener.bound(); // the resolved OS-chosen address (Option; absent when unknown) let token = listener.listen(); // the correlation token (the producer's; from the `listen` reply field) let sessions = listener.forwarded(); // the forwarded sessions, as they arrive -listener.close(self); // or Drop — tears the listen channel + far listener down +listener.close(self); // or Drop; tears the listen channel + far listener down ``` -- **`TunnelListener::open`** — the same `channels/tunnel/sub` open +- **`TunnelListener::open`**: the same `channels/tunnel/sub` open with `{resource, substrate}`; the establisher shape (bind-first vs accept-first) is the producer's per-registration choice, invisible to the constructor. What differs: bind-first replies carry - `"bound"` (ADR-008 §2 — the resolved address; informational, never - a dial target) and `"listen"` (ADR-008 Amendment 1 — the + `"bound"` (ADR-008 §2: the resolved address; informational, never + a dial target) and `"listen"` (ADR-008 Amendment 1: the correlation token the producer minted). -- **`bound()`** — `Option`-shaped: `Some` when the producer supplied +- **`bound()`**: `Option`-shaped: `Some` when the producer supplied the field (bind-first), `None` otherwise (accept-first, dial establishers, or old producers). reply#1-class fidelity (BND.ADDR). -- **`listen()`** — `Option`: the correlation token (bind-first - producers). The opener-side correlation registry — token → waiting - session — is assembly-side bookkeeping (consumer.md's posture: +- **`listen()`**: `Option`: the correlation token (bind-first + producers). The opener-side correlation registry, token → waiting + session, is assembly-side bookkeeping (consumer.md's posture: the protocol provides the op and params, not the wait state machine); `TunnelListener` hands the assembly the key. - **The forwarded side:** each accepted connection at the far @@ -135,7 +135,7 @@ listener.close(self); // or Drop — tears the listen channel + far li session, adopts the allocated channel (`TunnelSession::adopt`), and pumps against the local BIND client's halves. - **`peer` fidelity:** the forwarded op's params carry the accepted - connection's real remote address (reply#2-class fidelity — the + connection's real remote address (reply#2-class fidelity; the alksocks POC's sentinel retires). Producer-asserted metadata, not transport truth. - **Teardown:** dropping/`close`ing the listener EOFs the listen @@ -145,18 +145,18 @@ listener.close(self); // or Drop — tears the listen channel + far li ## The Data Plane (substrate-shaped) -- **Stream variant (`tcp`, `unix`):** `stream_halves()` — raw +- **Stream variant (`tcp`, `unix`):** `stream_halves()`: raw `AsyncRead`/`AsyncWrite` halves; the halves ARE the tunnel (raw pass-through, ADR-003). For owned access (spawning local pumps), `take_halves()` consumes the session's halves. - **Datagram variant (`udp`):** `send_datagram(&[u8])` / `recv_datagram() -> Option` over the mandatory `[len: u16 BE]` codec (ADR-003). `recv_datagram` returns - `Some(bytes)` per datagram — possibly empty (`len=0` is a legal - empty datagram) — and `None` only on stream EOF (the channels-level + `Some(bytes)` per datagram (possibly empty; `len=0` is a legal + empty datagram), and `None` only on stream EOF (the channels-level sentinel; the codec never collides with it, F-2). A >65535-byte send is rejected at frame time (`Oversize`). `take_halves()` on a - datagram session yields the RAW halves — the wire bytes carry the + datagram session yields the RAW halves; the wire bytes carry the codec framing (ADR-005's N-3 amendment: one `take_halves` semantic for every substrate; keep the session for codec-applied I/O, or compose the framed adapter at your own boundary). @@ -165,27 +165,27 @@ listener.close(self); // or Drop — tears the listen channel + far li One session = one channel = one tunnel. The session owns: -- **The adopted channel entry** — nothing upstream awaits the +- **The adopted channel entry**: nothing upstream awaits the adopter's pump (the W3 gap); the session closes it structurally. -- **The session-owned pump handle** (when the session spawned one — - the reverse path's `pump_against`). +- **The session-owned pump handle** (when the session spawned one, + i.e. the reverse path's `pump_against`). Teardown API: -- `close(self)` — abort the pump (if session-owned), tear down the +- `close(self)`: abort the pump (if session-owned), tear down the adopted channel (`ChannelManager::teardown_channel`), consume the session. The ungraceful path. -- `join(self)` — await pump completion (both directions finished; - half-close semantics fall out of `pump_bidi` — one direction EOFs, +- `join(self)`: await pump completion (both directions finished; + half-close semantics fall out of `pump_bidi`: one direction EOFs, the opposite sink shuts down, the other keeps pumping until its own EOF; W4-validated), THEN reap the adopted channel. Returns the `(u64, u64)` copy counts for observability. The graceful path. **Pump-less sessions** (after `take_halves`, or a datagram session - the caller drives directly): there is no session-owned pump — + the caller drives directly): there is no session-owned pump, so `join` completes immediately (nothing to await) and reaps only; the copy counts are `(0, 0)`. -- `Drop` — aborts the pump and sync-reaps the channel (teardown - semantics of `close`, best-effort — `Drop` cannot await). Dropping +- `Drop`: aborts the pump and sync-reaps the channel (teardown + semantics of `close`; best-effort, since `Drop` cannot await). Dropping without close/join never leaks the channel entry. - **`TunnelSession` does not implement `Clone`.** Aliasing a session would alias its teardown; multi-channel consumers hold a @@ -195,7 +195,7 @@ Teardown API: A peer-initiated `channel/close` (the generic channel op on the producer's serving registry) tears the SERVING side down; the -consumer's session still owns its adopted entry — `close`/`join` reaps +consumer's session still owns its adopted entry, so `close`/`join` reaps it. The reverse POC pinned this split (the close ran on the worker; the hub reaped its own entry, W3). EOF propagation is symmetric: either side's teardown reaches the other as the channels-level @@ -208,20 +208,20 @@ roles (alkcall ADR-022 §2): a worker dials the hub (connect side), serves its open ops (serving side), and its own outbound calls still resolve while serving (reverse POC: worker_outbound_calls_still_ resolve_while_serving). The consumer half adds nothing to the -connection's serving posture — a pure consumer (`from_connection`) +connection's serving posture: a pure consumer (`from_connection`) never dispatches inbound requests; a reverse-flow initiator needs no serving registry of its own. ## Open Questions -- **OQ-TN-12**: hub re-produce composition (deferred — a hub proxy +- **OQ-TN-12**: hub re-produce composition (deferred; a hub proxy re-exporting a consumed resource would add a `register_openable_with_establisher` whose establisher opens a consumer channel; no concrete consumer yet. The re-produce policy - sentence is pinned: the hub's ACL governs the final dial — ADR-006 + sentence is pinned: the hub's ACL governs the final dial, per ADR-006 Amendment 1) -- **OQ-TN-13**: UDP truncation (resolved — fail-loud, ADR-003) -- **OQ-TN-15/16**: resolved (ADR-007/008 — `open_direct`, +- **OQ-TN-13**: UDP truncation (resolved, fail-loud, ADR-003) +- **OQ-TN-15/16**: resolved (ADR-007/008: `open_direct`, `TunnelListener`, the forwarded op) ## References @@ -232,8 +232,8 @@ serving registry of its own. `ReverseTunnel` open_and_pump / close / join_and_reap; W3, W4) - `docs/research/tunnels-graduation.md` (the graduation research) - Upstream asks (review 008): `alkcall/docs/reviews/ - 008-tunnels-graduation-upstream-review.md` — landed in alkcall + 008-tunnels-graduation-upstream-review.md`; landed in alkcall 0.8.0 -- alkcall ADR-047 §5 (allocation — the initiator adopts), ADR-050 +- alkcall ADR-047 §5 (allocation; the initiator adopts), ADR-050 (`pump_bidi`), ledger CF-005/CF-006 (identity on the reverse path) \ No newline at end of file diff --git a/docs/architecture/decisions/001-open-params-layout.md b/docs/architecture/decisions/001-open-params-layout.md index a70676b..1c992ab 100644 --- a/docs/architecture/decisions/001-open-params-layout.md +++ b/docs/architecture/decisions/001-open-params-layout.md @@ -1,8 +1,8 @@ -# ADR-001: Open-Op Params Layout — Resource + Substrate +# ADR-001: Open-Op Params Layout (Resource + Substrate) ## Status -Accepted (2026-09-07); amended 2026-09-16 (params layout is per-op — +Accepted (2026-09-07); amended 2026-09-16 (params layout is per-op: the direct op pins its own params, see Amendment 1) ## Context @@ -13,7 +13,7 @@ carrier of "what to open." alknet ADR-071 §ALPN table noted `alknet/tunnel` as `[0, 1]` data in/out only; the addressing scheme was never decided there. Phase 0 resolved the framing of the problem (the OQ-TN-01 reframe, 2026-09-05): the rich "remote addressing" -framing was an XY problem — except in the `-D`/dynamic case (which +framing was an XY problem. Except in the `-D`/dynamic case (which composes at the assembly layer), a tunnel is either TCP or UDP, and `params` need only *identify a produced resource*, not carry a general-purpose address. @@ -21,7 +21,7 @@ general-purpose address. Constraints that shaped the decision: - `params` is ALPN-specific JSON, interpreted by the open op's - establisher/handler — never by the channels layer (alkcall + establisher/handler, and never by the channels layer (alkcall ADR-039). The layout is a one-way door once a consumer exists (wire-stable once published; additive changes only). - The identifier must be substrate-extensible without format changes @@ -34,7 +34,7 @@ Constraints that shaped the decision: address. A `host:port` in params would leak the producer's topology into the wire and pin the wire to address-shaped targets. - alkcall's registry schema-validates the open op's input - (`input_schema` runs before the establisher — review 007 Part B), + (`input_schema` runs before the establisher, per review 007 Part B), so the layout must be schema-describable (no per-substrate polymorphic payloads in v1). - Both POCs used exactly this shape (`TunnelParams {resource, @@ -52,21 +52,21 @@ JSON object: } ``` -- **`resource`** (string, required) — the produced resource +- **`resource`** (string, required): the produced resource identifier: the producer's stable name for the tunnel target. NOT an address; the producer's registry (an assembly-side construct, OQ-TN-11) maps the name to its backing (a local port, a docker container's port, an in-process service, a unix socket path). -- **`substrate`** (string, required) — the extensible discriminator: +- **`substrate`** (string, required): the extensible discriminator: `"tcp"` | `"udp"` | `"unix"` in v1. A new substrate is a new value, - not a format change — the same additive property SSH gets from + not a format change, the same additive property SSH gets from channel-type strings and SOCKS5 gets from ATYP values. The discriminator also selects the data-plane framing (ADR-003): the establisher and both pumps must agree on it, so it rides the open op rather than being inferred. - The open op's `input_schema` pins both fields as required strings; `substrate` is enum-validated against the v1 set. Unknown - substrate values are a schema failure (typed open error) — a + substrate values are a schema failure (typed open error): a consumer speaking a newer substrate to an older producer fails loudly, not silently. - Substrate-specific detail (the path of a unix socket, the UDP @@ -75,12 +75,12 @@ JSON object: - The open-failure error path rides alkcall ADR-049 (typed `channel:open_failed` with `details.reason` ∈ `dial_failed` / `unknown_resource` / `resource_shortage` / `handler_error` / - `timeout`): an unknown resource name is `unknown_resource` — no + `timeout`): an unknown resource name is `unknown_resource`; there is no params-level error field, no establishment frame (the pre-ADR-049 OQ-TN-09 hunch is superseded). Rich in-band addressing (SOCKS5 ATYP, per-datagram remote addresses) -enters only through the `-D`/dynamic-target composition path — it is +enters only through the `-D`/dynamic-target composition path; it is carried INSIDE the tunnel payload by whatever protocol the tunnel carries (socks5, udpgw-style framing), never in base params. @@ -90,7 +90,7 @@ carries (socks5, udpgw-style framing), never in base params. renames are a breaking wire change (do not). Adding an optional field is additive-safe; adding a `substrate` value is consumer-progressive (an older producer rejects with - `invalid_input`-class schema error — the SSH "unknown channel type" + `invalid_input`-class schema error, in the SSH "unknown channel type" posture, loud not silent). - **The BAST question is moot for params.** Params is JSON inside the channels open op (which is already a JSON envelope); the data @@ -101,7 +101,7 @@ carries (socks5, udpgw-style framing), never in base params. - **Producers own a naming discipline.** Resource ids are producer-scoped names (OQ-TN-11: collision domain = the producer's registry). Consumers learn available resources through the - ACL-filtered ops listing (OQ-TN-08) — the spec's + ACL-filtered ops listing (OQ-TN-08), so the spec's `OperationSpec.description` SHOULD carry a human-readable hint (e.g. "postgres on the worker's tailnet") since it round-trips through discovery. @@ -125,10 +125,10 @@ carries (socks5, udpgw-style framing), never in base params. The `{resource, substrate}` layout pins the `channels/tunnel/sub` open op. It is **per-op, not per-ALPN**: ADR-007 adds the `channels/tunnel/direct` op on the same `alk/tunnel` ALPN with its own -params (`{substrate, target}` — a dynamic target, a different -capability with its own scope; the resource-not-address invariant is -unchanged because a dynamic target is not a produced resource). The -per-op reading was always latent in the layout's basis — params are + params (`{substrate, target}`, i.e. a dynamic target, a different + capability with its own scope; the resource-not-address invariant is + unchanged because a dynamic target is not a produced resource). The + per-op reading was always latent in the layout's basis, since params are "the `input` of the operation" (alkcall ADR-047), and ops are per-flavor once flavor-form op ids exist (ADR-002 Amendment 1). Additive-only evolution per op stands: this amendment adds nothing to diff --git a/docs/architecture/decisions/002-alpn-strategy.md b/docs/architecture/decisions/002-alpn-strategy.md index 278f1c6..70ccfd0 100644 --- a/docs/architecture/decisions/002-alpn-strategy.md +++ b/docs/architecture/decisions/002-alpn-strategy.md @@ -25,7 +25,7 @@ enum) and the ALPN namespace (a published string is wire-stable). Prior art surveyed in Phase 0 (`ssh-socks5-survey.md`): -- SSH uses ONE channel mechanism for all forwarding flavors — the +- SSH uses ONE channel mechanism for all forwarding flavors; the channel type string (`direct-tcpip`, `forwarded-tcpip`, `direct-streamlocal`) is per-open metadata on one transport, not a separate transport per flavor. @@ -35,7 +35,7 @@ Prior art surveyed in Phase 0 (`ssh-socks5-survey.md`): stream (length-framed datagrams inside a TCP tunnel). - With OQ-TN-02 resolved as endpoint-at-open for base UDP resources, the substrate discriminator in `params` tells the establisher and - pumps which framing to expect — exactly option A's shape. The + pumps which framing to expect, exactly option A's shape. The discriminator is load-bearing (it selects the framing, ADR-003), so it must ride the open op; the ALPN carries no substrate information at all. @@ -49,16 +49,16 @@ pump handler, and consumer session type are shared across substrates session types). - The open op id is `channels/tunnel/sub` (the `channels//sub` - convention of alkcall ADR-047 — the open op is a `Sub`-typed + convention of alkcall ADR-047 (the open op is a `Sub`-typed operation: the reply carries `channel_id` once, the data plane flows on the channel's `BiStream`). - The channel's ALPN marker on the open-op spec is `alk/tunnel` - (`ChannelOpenSpec::new("alk/tunnel")`) — the value the + (`ChannelOpenSpec::new("alk/tunnel")`, which is the value the per-ALPN dispatch and the adopted-side manager record for observability. - If a future tunnel flavor needs structurally different framing from byte zero with no params-dependent dispatch, that is a NEW ALPN - decided by a new ADR before its first consumer — never a change to + decided by a new ADR before its first consumer, and never a change to `alk/tunnel`'s meaning. ALPN strings are wire-stable once published. @@ -71,12 +71,12 @@ session types). `(resource, substrate)`. - **Framing is params-dependent, not ALPN-dependent.** The establisher validates `substrate` semantically (the registry lookup); the pump - handler is framing-agnostic (raw pass-through halves — the codec + handler is framing-agnostic (raw pass-through halves; the codec lives on the consumer/producer edge for UDP, ADR-003). A mismatched `substrate` value fails at schema/establisher time, loudly. - **ALPN namespace hygiene:** `alk/tunnel` is published here. The `alknet/tunnel` spelling in the alknet ADR-071 table predates the - `alknet/` → `alk/` prefix swap (alkcall v0.1.1) — docs and code + `alknet/` → `alk/` prefix swap (alkcall v0.1.1), so docs and code must not perpetuate the old prefix. ## References @@ -88,20 +88,20 @@ session types). ## Amendment 1 (flavor-form op ids, 2026-09-16) -The decision stands — one ALPN, no ALPN split per flavor. What +The decision stands: one ALPN, no ALPN split per flavor. What changes: the one-ALPN rule is realized as **one ALPN, several `Sub`-typed open ops**, because ADR-007 (the direct op) and ADR-008 (the forwarded op) are different capabilities sharing the tunnel -transport — ssh's one-transport-many-channel-types shape. +transport, ssh's one-transport-many-channel-types shape. - The op-id convention extends to flavor form: `channels//sub` (the original resource op, reserved and unchanged), `channels//direct` (dial an arbitrary target), and `channels//forwarded` (accept-as-open toward the listen - opener). New flavors are new op ids — additive; the `…/sub` op is + opener). New flavors are new op ids, which is additive; the `…/sub` op is never reused for a different meaning. -- The alternative — sibling ALPNs (`alk/tunnel-direct`, - `alk/tunnel-forwarded`) with standard `…/sub` op ids — was rejected: +- The alternative, sibling ALPNs (`alk/tunnel-direct`, + `alk/tunnel-forwarded`) with standard `…/sub` op ids, was rejected: it fragments the tunnel namespace per flavor, forces hub relay to track three ALPNs, and amends this ADR's own new-ALPN justification clause (the flavors share framing; none needs structurally different @@ -109,7 +109,7 @@ transport — ssh's one-transport-many-channel-types shape. - Upstream coordination: alkcall ADR-047 pins discovery's ALPN derivation to exactly the `…/sub`/`…/pub` suffix shapes (`derive_alpn_from_op_name`); flavor-form op ids need that - derivation (and the hub-relay reconstruction riding it) extended — + derivation (and the hub-relay reconstruction riding it) extended; it was filed upstream (`alkcall/docs/reviews/008-tunnels-graduation-upstream-review.md`, ask #1) before the first consumer. Runtime was never blocked: diff --git a/docs/architecture/decisions/003-codec-and-udp-framing.md b/docs/architecture/decisions/003-codec-and-udp-framing.md index fffb2e6..26cb733 100644 --- a/docs/architecture/decisions/003-codec-and-udp-framing.md +++ b/docs/architecture/decisions/003-codec-and-udp-framing.md @@ -1,11 +1,11 @@ -# ADR-003: Data-Plane Codec — Raw Pass-Through (Stream) / Mandatory Length Framing (UDP) +# ADR-003: Data-Plane Codec, Raw Pass-Through (Stream) / Mandatory Length Framing (UDP) ## Status Accepted (2026-09-07; supersedes the Phase 0 "codec direction" hunch -by making the UDP framing mandatory — reverse POC finding F-2). +by making the UDP framing mandatory, per reverse POC finding F-2). Amended (2026-09-08, review 001 U-1 remediation): the substrate- -boundary placement is pinned mechanically — the `local` feature's +boundary placement is pinned mechanically, so the `local` feature's `UdpHalf` composes the codec at the boundary (channel side framed, target side raw), with the pump-window and recv-scratch disciplines the composition requires. See §Amendment (2026-09-08). @@ -14,10 +14,10 @@ the composition requires. See §Amendment (2026-09-08). The tunnel data plane rides a channels data channel: raw bytes inside the channel `BiStream` (the channels layer strips its 8-byte header -transparently — alknet ADR-093/ADR-071; AGENTS.md convention 5). The +transparently, per alknet ADR-093/ADR-071 and AGENTS.md convention 5). The tunnel protocol owns whatever framing it puts inside the `BiStream`. A tunnel has one data stream per direction, so there is no -sub-demux key — a 5-byte header like alktty's (with `stream_type`) +sub-demux key (a 5-byte header like alktty's, with `stream_type`, would be pure overhead here. The Phase 0 direction (`phase-0-findings.md` checklist, 2026-09-06): @@ -29,14 +29,14 @@ Two validation passes sharpened this: 1. **Forward POC (2026-09-06, 17 tests):** the codec round-trips datagrams split across chunks, batched in one chunk, partial - headers, and empty datagrams (`len=0` is a legal payload, NOT EOF - — EOF is the channels-level `length=0` chunk sentinel; the two + headers, and empty datagrams (`len=0` is a legal payload, NOT EOF; + EOF is the channels-level `length=0` chunk sentinel; the two coexist at different layers). The 1400-byte (max ethernet MTU payload) datagram rides the bounded-buffer path; >65535 is rejected at frame time. 2. **Reverse POC finding F-2 (2026-09-07, executable-pinned):** in the RAW pass-through pump shape, an empty UDP datagram is a - zero-byte read from the substrate adapter — `tokio::io::copy` + zero-byte read from the substrate adapter, and `tokio::io::copy` treats `Ok(0)` as end-of-stream and shuts the pump down. An empty datagram and the zero-length EOF sentinel are the SAME wire shape at the pump level. The test pins the non-round-trip so the @@ -49,7 +49,7 @@ The data plane inside a `alk/tunnel` channel is: - **Stream substrates (`tcp`, `unix`): raw pass-through.** The halves are the tunnel; zero tunnel-level framing. The only wire overhead is the channels 8-byte chunk header. A zero-length read on a stream - substrate is genuinely EOF (stream semantics) — no collision. Both + substrate is genuinely EOF (stream semantics), so there is no collision. Both pumps run `alkcall::channels::pump_bidi` (ADR-050) over the raw halves. - **Datagram substrate (`udp`): mandatory `[len: u16 BE][datagram]` @@ -57,7 +57,7 @@ The data plane inside a `alk/tunnel` channel is: length-prefixed frame, on BOTH directions: - The **establisher** wraps the dialed UDP socket in the framed adapter BEFORE returning the plan (the plan payload is the - adapter, not a raw socket) — the pump handler stays + adapter, not a raw socket), so the pump handler stays substrate-agnostic (ADR-004's invariant; producer.md's placement). - Consumer side: the `TunnelSession`'s datagram variant does the same over the adopted halves. @@ -66,29 +66,29 @@ The data plane inside a `alk/tunnel` channel is: POC). EOF is exclusively the channels-level sentinel; the codec never emits zero-length reads, so the layers never collide. This is the load-bearing property F-2 validated from a second angle: - the length prefix makes an empty datagram two bytes — unambiguous + the length prefix makes an empty datagram two bytes, which is unambiguous with EOF. - **MTU discipline:** the codec rejects framing a >65535-byte - datagram (`Oversize` error at frame time, never a wire overflow — + datagram (`Oversize` error at frame time, never a wire overflow; the u16 length field would wrap). A truncated receive (caller buffer smaller than the datagram) fails loudly at the adapter - level — resolved 2026-09-07 (OQ-TN-13, fail-loud): silent + level, resolved 2026-09-07 (OQ-TN-13, fail-loud): silent truncation would corrupt the framing invariants (the same invariant class F-2 protects). - **Chunking is transparent.** Datagrams split across channel chunks, - batch into single chunks, and survive partial headers — the + batch into single chunks, and survive partial headers; the incremental decoder is the only consumer-visible decode path (forward POC: maximally awkward 7-byte chunk splits verified). ## Consequences -### Amendment (2026-09-08 — the boundary mechanics, from review 001 U-1's remediation) +### Amendment (2026-09-08, the boundary mechanics from review 001 U-1's remediation) The placement sentence ("the establisher wraps the dialed UDP socket in the framed adapter before returning the plan") leaves the adapter's *mechanics* open; review 001 found `UdpHalf` shipping as a raw -boundary-preserving adapter (the codec on one end only — real targets -saw the tunnel's length prefixes; empty datagrams collapsed to EOF at + boundary-preserving adapter (the codec on one end only, so real targets + saw the tunnel's length prefixes; empty datagrams collapsed to EOF at the pump's zero-read check; >8 KiB datagrams split at the copy buffer). The remediation pins the mechanics: @@ -98,29 +98,29 @@ buffer). The remediation pins the mechanics: sends each decoded payload as one raw datagram; `poll_read` **frames** (`frame_datagram`) each raw target datagram and hands frame bytes to the pump. "Frame on write, de-frame on read" describes the TARGET's - face and is the inverted (wrong) reading — the review's original + face and is the inverted (wrong) reading; the review's original fix sketch had it inverted and was corrected in re-validation. - **The write half accepts all offered bytes and reports them consumed.** The pump's copy engine re-offers a *full* window unsatisfied until it drains (tokio `copy`'s `read_done` path never tops the window up), so a partial-frame tail could sit forever in a - full window if the adapter accepted only complete frames — the + full window if the adapter accepted only complete frames; the >8 KiB frame would deadlock the pump. Completed datagrams queue for the socket; a trailing partial frame waits in the `DatagramReader` - (its state is bounded by construction — the u16 cap, review 001 + (its state is bounded by construction, by the u16 cap, review 001 N-2). Backpressure is an accepted-byte high-water mark on `poll_write`, and the socket drain pumps opportunistically on write/flush/shutdown (tokio's copy polls flush after every window drain). - **The recv scratch is full-size (65535) and lazy.** `try_recv` - truncates SILENTLY at the buffer size — a smaller scratch corrupts + truncates SILENTLY at the buffer size, and a smaller scratch corrupts the framing before any size check could reject it (the review's "size to the MTU" suggestion is unsafe as written). The 64 KiB eager allocation is instead lazy (first recv), the U-3 mitigation. - **A >caller-buffer frame is staged, not an error.** The frame's length prefix reaches the pump on the first read; the remainder hands out across reads. The pump's far-side de-framer reassembles - the true length — the raw adapter's read-level `InvalidData` is + the true length; the raw adapter's read-level `InvalidData` is replaced by staging (the fail-loud posture now protects the *frame stream*, and a wire-level over-bound datagram remains a loud `Oversize`-class error at the frame path). @@ -131,9 +131,9 @@ recording-target suite). ## Consequences (original) -- **TCP/unix tunnels have 0 B tunnel overhead** — the SSH +- **TCP/unix tunnels have 0 B tunnel overhead**, matching the SSH `direct-tcpip` experience (the payload is the bytes). -- **UDP correctness requires the codec** — raw pass-through over UDP +- **UDP correctness requires the codec**, since raw pass-through over UDP is structurally broken for empty datagrams (F-2). The substrate discriminator (ADR-001) is what tells both sides to engage the codec; a consumer bypassing it for UDP is a spec violation, not a @@ -141,8 +141,8 @@ recording-target suite). - **`tokio::io::copy`-shaped pumps fit UDP only through the framed adapter** (the `UdpHalf` + codec wrapper presents `AsyncRead`/`AsyncWrite` whose zero-length read never occurs for a - well-formed stream — `len=0` yields a 2-byte wire frame). -- **The BAST document exists** (AGENTS.md convention 12 — the + well-formed stream; `len=0` yields a 2-byte wire frame). +- **The BAST document exists** (AGENTS.md convention 12; the mandatory framing IS binary framing beyond pass-through, so the trigger fired): `docs/architecture/bast.md` describes the `[len: u16 BE]` datagram frame per the BAST meta-schema; the @@ -160,6 +160,6 @@ recording-target suite). sentinel layering + MTU); reverse POC: `docs/research/reverse-poc-summary.md` §F-2 (the mandate) - `docs/research/ssh-socks5-survey.md` §UDP (tun2proxy udpgw framing - precedent — no CONN_ID, no FLAGS; boundary framing only) + precedent (no CONN_ID, no FLAGS; boundary framing only) - alknet ADR-071/093 (channels wire format; the zero-length sentinel) - alkcall `channels::pump_bidi` (ADR-050) \ No newline at end of file diff --git a/docs/architecture/decisions/004-no-backend-trait.md b/docs/architecture/decisions/004-no-backend-trait.md index 1db5c42..a1fd4dd 100644 --- a/docs/architecture/decisions/004-no-backend-trait.md +++ b/docs/architecture/decisions/004-no-backend-trait.md @@ -1,4 +1,4 @@ -# ADR-004: No TunnelBackend Trait — Halves Functions at the Assembly Layer +# ADR-004: No TunnelBackend Trait (Halves Functions at the Assembly Layer) ## Status @@ -8,8 +8,8 @@ re-produce thread) ## Context alktty has a `TtyBackend` trait because backends (local PTY, docker, -SSH) produce complex handles — `TtyHandle` carries -stdin/stdout/stderr/exit-code quadruple + control — and the adapter +SSH) produce complex handles: `TtyHandle` carries +stdin/stdout/stderr/exit-code quadruple + control, and the adapter pumps them. The Phase 0 question (OQ-TN-05): does alktunnels need a `TunnelBackend` trait, or is the two-pump handler + feature-gated substrate modules the whole story? @@ -24,7 +24,7 @@ backend's: connection is the same boxed-halves shape. The UDP POC gave the executable answer for dial flows (2026-09-06): -`pump_halves` is generic over boxed `(AsyncRead, AsyncWrite)` halves — +`pump_halves` is generic over boxed `(AsyncRead, AsyncWrite)` halves: TCP contributes `into_split()` halves, UDP contributes the `UdpHalf` socket adapter, the pump never knows which. "Produce boxed halves for a resource" is a function, not a trait. @@ -33,7 +33,7 @@ The one remaining thread was the hub re-produce composition (does a hub proxy that re-produces a consumed resource need a trait or a composition helper?). The reverse POC (2026-09-07) resolved the mechanics: a hub proxy is a consumer + producer composed at the -assembly layer — the reverse POC's hub is exactly this shape (it +assembly layer; the reverse POC's hub is exactly this shape (it consumes a channels connection and produces reverse-tunnel opens toward the worker); re-exporting (registering an openable for a resource the hub itself consumes) would add a @@ -51,7 +51,7 @@ assembly layer: - **The produce function.** The producer's establisher resolves `(resource, substrate)` → boxed halves - (`Box` ×2 — the `+ Sync` is + (`Box` ×2; the `+ Sync` is the `ChannelPlan` bound, reverse POC F-1) or a framed UDP adapter. The dial itself is substrate code living in feature-gated backend modules (`local` feature: real sockets), injected at assembly: @@ -64,12 +64,12 @@ assembly layer: open op; its establisher pops the next accepted connection from an assembly-owned listener queue and returns it as the plan payload. The listener lifecycle (bind, accept loop, cancel) is assembly - code — the protocol never binds (OQ-TN-04). The reverse POC's + code; the protocol never binds (OQ-TN-04). The reverse POC's worker (dial) and the forward POC's producer (dial) both rode this shape; the listen establisher is the same plan-flow with a different halves source. - **The pump handler is substrate-agnostic by construction:** - `pump_bidi(bidi, t_read, t_write)` over the downcast plan payload — + `pump_bidi(bidi, t_read, t_write)` over the downcast plan payload; it cannot know whether the halves came from TCP, UDP, a unix socket, or an in-process pipe, and must not. - **Datagram substrates** contribute a framed adapter (ADR-003) @@ -78,9 +78,9 @@ assembly layer: every assembly layer re-implements the same glue (registry + dial + establisher construction + pump wiring) three or more times, extract a `TunnelBackend`-shaped trait THEN (the alknet ADR-078 - convergence-test threshold — extract when the shapes have converged + convergence-test threshold: extract when the shapes have converged across consumers, not before) - — as an additive layer, not a v1 wire/ABI commitment. Until then + and do it as an additive layer, not a v1 wire/ABI commitment. Until then the function-plus-closure shape keeps the crate dependency-light and wasm-clean. @@ -89,14 +89,14 @@ assembly layer: - **The default crate stays wasm-clean** with no socket/platform deps; `local` (sockets) and future backends (docker, process) are feature-gated modules implementing dial/listen functions. -- **Assembly layers write more glue than a trait would save** — the - accepted cost, validated twice (both POCs' producers are ~200 lines +- **Assembly layers write more glue than a trait would save**, which is + the accepted cost, validated twice (both POCs' producers are ~200 lines including the UDP adapter). - **Plan payloads are `Send + Sync`** (reverse POC F-1, documented on alkcall's `ChannelPlan` type in 0.7.0): socket-backed handles carry `+ Sync` naturally; non-Sync handles (process pipes as boxed trait objects) need a wrapper. The spec documents the constraint. -- **The listen variant needs no new wire surface** — same open op, +- **The listen variant needs no new wire surface**: same open op, same params, same typed errors (an empty listener queue during establishment is `resource_shortage`; a closed listener is `dial_failed`-class). @@ -108,8 +108,8 @@ assembly layer: - Forward POC: `docs/research/poc-summary.md` §5 (halves-not-trait); reverse POC: `docs/research/reverse-poc-summary.md` (the hub shape, F-1) -- alktty ADR-002 (`TtyBackend` — the contrast case: when a trait IS - warranted), alktty ADR-003 (backend placement — the feature-gate +- alktty ADR-002 (`TtyBackend`, the contrast case: when a trait IS + warranted), alktty ADR-003 (backend placement, the feature-gate pattern) - AGENTS.md conventions 4/7/14 (wasm-clean, substrate-agnostic, feature flags) \ No newline at end of file diff --git a/docs/architecture/decisions/005-consumer-session-owns-teardown.md b/docs/architecture/decisions/005-consumer-session-owns-teardown.md index 9240ad9..587393b 100644 --- a/docs/architecture/decisions/005-consumer-session-owns-teardown.md +++ b/docs/architecture/decisions/005-consumer-session-owns-teardown.md @@ -4,7 +4,7 @@ Accepted (2026-09-07; resolves the reverse POC W3 finding). Amended (2026-09-08, review 001 N-3 decision): the datagram -`take_halves` posture is pinned — raw framed halves stay the only +`take_halves` posture is pinned: raw framed halves stay the only owned-halves shape (`take_halves` keeps its semantics for every substrate), the datagram accessors remain the codec path, and a framed-halves convenience is deferred to the first consumer that @@ -17,23 +17,23 @@ A tunnel channel's two ends have asymmetric lifecycle machinery: - **The serving side (the side the open op ran on)** is wrapper-managed: the open wrapper awaits the pump handler's `JoinHandle`, and its completion triggers channel teardown (drop of - the demux sender = EOF to the handler's read half — alkcall ADR-049 + the demux sender = EOF to the handler's read half, per alkcall ADR-049 + review 007 R-02). Out-of-band `channel/close` also tears it down. Nothing leaks. - **The adopting side (the side that called the open op and adopted - the channel ID — the consumer)** has NO such machinery: + the channel ID, i.e. the consumer)** has NO such machinery: `ChannelManager::adopt_channel` installs routing state nothing awaits. The consumer-side pump (`pump_bidi` spawned locally) is somebody's `JoinHandle`; the adopted channel state is nobody's to reap. If the assembly layer drops both, the channel entry leaks in - the manager until transport EOF (`clear_all`) — and worse, a + the manager until transport EOF (`clear_all`); and worse, a consumer that drops the pump handle mid-flight aborts the pump without reaping, leaving half-open state on the peer. The reverse POC found this by construction (W3, 2026-09-07): the hub -must hold its pump handle and reap (`teardown_channel`) itself — +must hold its pump handle and reap (`teardown_channel`) itself. `ReverseTunnel::join_and_reap`/`close` in the POC. The lifecycle is -assembly-layer by design (OQ-TN-04 — no forced binding means no +assembly-layer by design (OQ-TN-04: no forced binding means no protocol-owned listener lifecycle either), but the POC demonstrated that leaving teardown discipline to each assembly layer's memory is exactly the kind of gap the spec must close with a type, not a doc @@ -49,31 +49,31 @@ plane (raw halves vs datagram codec, ADR-003). The consumer half is a typed session, **`TunnelSession`**, and it owns its teardown: -- **Construction:** `TunnelSession::open(client, params)` — calls the +- **Construction:** `TunnelSession::open(client, params)`: calls the open op (`ChannelClient::open_channel` on the forward path; the reverse path's `open_reverse_channel` + `adopt_channel` equivalent), adopts the returned channel ID, splits the channel `BiStream`, and presents the substrate-shaped data plane: - - **Stream variant:** raw halves (`AsyncRead`/`AsyncWrite`) — the + - **Stream variant:** raw halves (`AsyncRead`/`AsyncWrite`); the halves ARE the tunnel. - **Datagram variant:** `send_datagram`/`recv_datagram` over the - mandatory codec (ADR-003) — boundary-preserving, `len=0` is an + mandatory codec (ADR-003), which is boundary-preserving; `len=0` is an empty datagram, `recv_datagram` returns `None` only on stream EOF. - **Pump ownership:** the session's `pump_against(accepted_halves)` (the reverse-flow use) spawns `pump_bidi` and holds the returned - `JoinHandle`. For the pure consumer (no local pump — the session + `JoinHandle`. For the pure consumer (no local pump; the session halves ARE handed to the caller), the session does not spawn; the caller drives the halves and the session still owns the channel entry. - **Teardown API (the point of the ADR):** - - `close(self)` — abort the pump (if session-owned), tear down the + - `close(self)`: abort the pump (if session-owned), tear down the adopted channel (`teardown_channel`), consume the session. The abort path (ungraceful). - - `join(self)` — await pump completion (both directions finished), + - `join(self)`: await pump completion (both directions finished), THEN reap the adopted channel, return the `(u64, u64)` copy counts from `pump_bidi` for observability. The graceful path. - Dropping the session without either: the `Drop` impl tears the - channel down (the panic-free fallback — never leak the entry). + channel down (the panic-free fallback, never leaking the entry). `Drop` cannot await, so it aborts the pump and calls the sync `teardown_channel`; this is semantically `close`. - **Half-open semantics fall out of `pump_bidi`** (ADR-050): one @@ -81,7 +81,7 @@ its teardown: crosses the mux) → the other pump keeps running until its own EOF. The session's `join` completes when BOTH pumps finish. Half-close semantics validated end-to-end (reverse POC W4). -- **Error surface:** a failed open resolves a typed error — +- **Error surface:** a failed open resolves a typed error: `ChannelOpenError::CallFailed` carrying the wire `CallError`; branch on `establishment_reason()` for `channel:open_failed`'s reason (ADR-049 §4). A failed open never yields a session (no @@ -92,19 +92,19 @@ its teardown: ## Consequences -### Amendment (2026-09-08 — the N-3 datagram `take_halves` posture, decided before the first external consumer) +### Amendment (2026-09-08, the N-3 datagram `take_halves` posture, decided before the first external consumer) Review 001 N-3: a datagram session exposes codec-applied accessors -(`send_datagram`/`recv_datagram`) until `take_halves` — then the +(`send_datagram`/`recv_datagram`) until `take_halves`, at which point the caller owns RAW halves whose wire bytes carry the `[len: u16 BE]` framing. Documented, but a footgun asymmetry the API window (this ADR's surface becomes ABI-stable at the first external -consumer) forced a decision on. Decided: **option 1 — keep as-is, -carried by docs**, with the rationale: +consumer) forced a decision on. Decided: **option 1 (keep as-is, +carried by docs)**, with the rationale: -- **No external consumer exists** (verified 2026-09-08 — no crate in +- **No external consumer exists** (verified 2026-09-08; no crate in the workspace depends on alktunnels); the window is open, so a - breaking reshape was *available* — but nothing yet demonstrates a + breaking reshape was *available*, but nothing yet demonstrates a need for a different shape. - **The framing asymmetry is ADR-003's spec, not drift.** The datagram data plane is mandatory-framing by substrate @@ -113,23 +113,23 @@ carried by docs**, with the rationale: boundary." `take_halves` yielding raw halves keeps ONE semantic across substrates (the tunnel's wire bytes, uninterpreted); a per-substrate `take_halves` (option 3) diverges semantics by - substrate — the worse confusion at the type level even as it + substrate (the worse confusion at the type level even as it removes the byte-level footgun. - **`pump_against` needs the framed adapter, not framed halves.** The adjacent seam (N-11, resolved in-review with the doc note): a UDP session pumped against an accepted local half requires the FRAMED - adapter on the accepted side — the `local` feature's `UdpHalf` (or + adapter on the accepted side, namely the `local` feature's `UdpHalf` (or an assembly equivalent), not a raw socket. What composes there is the adapter AT the boundary, not a session-side wrapper pair; a `take_framed_halves` convenience would not feed this use. - **The convenience is additive and can land later.** Option 2's `take_framed_halves` breaks nothing when added after the first consumer (unlike a `take_halves` reshape), so its timing is - decoupled from this window — deferring it costs nothing now, and + decoupled from this window, so deferring it costs nothing now, and the first real consumer's reach defines its shape (a `DatagramReader`-wrapped pair has an open design question: an `AsyncRead` face re-collides with the F-2 empty-datagram/EOF - ambiguity the codec exists to prevent — the codec-side framing + ambiguity the codec exists to prevent: the codec-side framing wants accessor-shaped surface, which the session already has). - **The docs carry the contract.** `take_halves`'s doc states the framing (landed with the implementation); consumer.md's datagram @@ -138,14 +138,14 @@ carried by docs**, with the rationale: **Consequence:** datagram callers wanting codec-applied I/O after taking ownership should keep the session (drive `send_datagram`/ `recv_datagram`; the session IS the owner) or compose `UdpHalf` at -their own boundary — `take_halves` is the raw-wire escape hatch and +their own boundary; `take_halves` is the raw-wire escape hatch and stays documented as such. Revisit only on a demonstrated consumer need (file an OQ at that point). ### Consequences (original) - **Assembly layers cannot leak adopted channels** by forgetting to - reap — the type is the discipline (the W3 gap closes structurally). + reap; the type is the discipline (the W3 gap closes structurally). - **The reverse-flow initiator gets the same session shape** as the forward consumer: the reverse POC's hub-side `ReverseTunnel` is the seed; the spec generalizes it so `-L` and `-R` consumers share one @@ -156,7 +156,7 @@ need (file an OQ at that point). already wrapper-managed (R-02); the session only fixes the adopter asymmetry. - **`Drop`-based teardown is best-effort** (abort + sync reap, no - await) — the documented contract; graceful flows call `join` or + await): the documented contract; graceful flows call `join` or `close` explicitly. ## References @@ -165,7 +165,7 @@ need (file an OQ at that point). finding), §W4 (half-close validation) - OQ-TN-03, OQ-TN-04, OQ-TN-09 (promoted) - alkcall ADR-049 + review 007 R-02 (wrapper-managed serving side), - ADR-050 (`pump_bidi` + copy counts), ADR-047 §5 (allocation — the + ADR-050 (`pump_bidi` + copy counts), ADR-047 §5 (allocation; the initiator adopts) -- AGENTS.md convention 10 (limits inherited — the session adds no +- AGENTS.md convention 10 (limits inherited; the session adds no second bookkeeping layer; it owns exactly one channel's lifecycle) \ No newline at end of file diff --git a/docs/architecture/decisions/006-access-control-posture.md b/docs/architecture/decisions/006-access-control-posture.md index 6211c8a..dad446e 100644 --- a/docs/architecture/decisions/006-access-control-posture.md +++ b/docs/architecture/decisions/006-access-control-posture.md @@ -1,8 +1,8 @@ -# ADR-006: Access-Control Posture — the Open Gate Is the Security Boundary +# ADR-006: Access-Control Posture: the Open Gate Is the Security Boundary ## Status -Accepted (2026-09-07; records the OQ-TN-08 resolution as a decision — +Accepted (2026-09-07; records the OQ-TN-08 resolution as a decision; the posture was resolved in Phase 0 but the consumer-visible constants had no ADR backing); amended 2026-09-16 (scope separation for the direct/forwarded ops + the hub re-produce policy sentence, see @@ -11,7 +11,7 @@ Amendment 1) ## Context Tunnels reach local networks: opening a tunnel channel instructs the -producer to dial (or accept for) a target — potentially anything the +producer to dial (or accept for) a target, potentially anything the producer's host can reach. The open gate is therefore the security boundary of this crate, in the same sense that SSH's `AllowTcpForwarding`/`PermitOpen` govern `direct-tcpip` opens. The @@ -20,13 +20,13 @@ this crate vs alkcall's op-level ACL vs the assembly layer. Phase 0 resolved the conceptual tangle (2026-09-05, the OQ-TN-08 resolution): at protocol-crate level, a produced resource belongs to -the far side of the connection — the protocol works under that +the far side of the connection, and the protocol works under that assumption, so the ACL story is exactly alkcall's existing op-level ACL. No new policy layer. The hub/overlay mechanism (workers connecting in, the hub re-exposing resources per ITS policy) is a downstream, assembly-layer concern (OQ-TN-12's scope). -What was left implicit — and needs pinning, because it is +What was left implicit, and needs pinning because it is consumer-visible and stable-once-published: - The scope constant governing tunnel opens (`tunnel:open`) is @@ -39,10 +39,10 @@ consumer-visible and stable-once-published: ## Decision 1. **The open gate is the boundary, and it is alkcall's ACL.** The - open op's `AccessControl` rides `ChannelCore::register_openable` — + open op's `AccessControl` rides `ChannelCore::register_openable`; the registry runs the ACL before the establisher. The base gate is the scope constant **`TUNNEL_OPEN_SCOPE = "tunnel:open"`** - (required scope, exact string — stable once published; consumers + (required scope, exact string; stable once published, and consumers will request grants by it). An identity without the scope is `FORBIDDEN` before any establisher code runs. 2. **The scope string is wire/ACL-stable** from the first consumer: @@ -51,7 +51,7 @@ consumer-visible and stable-once-published: 3. **No target allowlists and no tunnel-specific policy machinery in v1.** The resource is owned by the far side (the Phase 0 resolution); a producer that wants finer granularity wires - `OwnershipProvider` checks into ITS establisher — the protocol + `OwnershipProvider` checks into ITS establisher; the protocol provides the seam (the establisher sees the per-call opener identity, alkcall 0.7.0 CF-006), not the policy. This mirrors alktty's posture (`TTY_OPEN_SCOPE` + op-level ACL, no @@ -80,17 +80,17 @@ consumer-visible and stable-once-published: ## Consequences - **One security knob per producer deployment** (the op-level ACL + - scope grants) instead of a tunnel-specific policy language — the - accepted simplicity; per-resource granularity is an establisher + scope grants) instead of a tunnel-specific policy language, which is + the accepted simplicity; per-resource granularity is an establisher customization (the seam), not a v1 surface. - **The scope constant is a compatibility commitment:** treat `tunnel:open` like the ALPN string. Changing it after consumers request grants is a breaking ACL change. -- **No allowlist concept** — a wildcard-egress producer +- **No allowlist concept**: a wildcard-egress producer (`0.0.0.0/0`-style egress) is expressed by registering a resource whose backing is that egress; gating it is the assembly layer's ownership check at the seam. The protocol stays out of the business. -- **Tunnels reach local networks** — producers MUST treat the gate as +- **Tunnels reach local networks**, so producers MUST treat the gate as the security boundary; the spec docs state this posture (producer.md). ## References @@ -99,7 +99,7 @@ consumer-visible and stable-once-published: - alkcall ADR-047 (open ops + ACL wiring), ADR-017 §7 (the `auth_token` path), ledger CF-005/CF-006 (identity precedence, per-call opener); alknet ADR-024/ADR-050 (registry layering, - ownership — the alkcall ports ADR-019/ADR-011) + ownership; the alkcall ports ADR-019/ADR-011) - alktty's `TTY_OPEN_SCOPE` posture (the sibling precedent) - AGENTS.md convention 13 (access control) @@ -109,19 +109,19 @@ The posture stands; the scope space grows with the ops (ADR-007, ADR-008): - **`TUNNEL_DIRECT_SCOPE = "tunnel:direct"`** gates - `channels/tunnel/direct` — dynamic-target egress. **Never implied by + `channels/tunnel/direct` for dynamic-target egress. **Never implied by `tunnel:open`**: "dial `postgres-primary`" and "dial anything from my network" are different capabilities, and the separation is structural (scopes attach to op specs, not params). A deployment granting both grants both explicitly. - **`TUNNEL_FORWARDED_SCOPE = "tunnel:forwarded"`** gates - `channels/tunnel/forwarded` on the opener side — the grant for + `channels/tunnel/forwarded` on the opener side; it is the grant for RECEIVING producer-initiated forwarded connections (ingress). Ingress and egress are different grants (the alksocks POC #5 `socks5:bind` gate proved the distinction matters in practice). - **Hub re-produce policy sentence (the security model):** a hub that re-produces the direct op applies ITS OWN target policy at the - re-produce hop — the hub's ACL, not the original opener's, governs + re-produce hop; the hub's ACL, not the original opener's, governs the final dial. The hub is the network boundary; the original opener's grant gets it to the hub, the hub's own grant decides what it will dial on the far side. Correct-by-default; pinned here diff --git a/docs/architecture/decisions/007-direct-open-op.md b/docs/architecture/decisions/007-direct-open-op.md index 5907493..03377a8 100644 --- a/docs/architecture/decisions/007-direct-open-op.md +++ b/docs/architecture/decisions/007-direct-open-op.md @@ -1,4 +1,4 @@ -# ADR-007: The Direct Open Op — Dynamic-Target Egress +# ADR-007: The Direct Open Op (Dynamic-Target Egress) ## Status @@ -11,7 +11,7 @@ the backing; rich addressing rides the `-D`/dynamic composition path, never the wire. That decision stands. The use case that justifies an *additive* revisit is the one the alksocks graduation research (`docs/research/tunnels-graduation.md`) named: a consumer asking the -producer to dial an arbitrary target chosen at session time — ssh's +producer to dial an arbitrary target chosen at session time, i.e. ssh's `direct-tcpip` shape, SOCKS5 CONNECT's egress, and the vpn-like dial-callback translation all need it. A dynamic target is a different capability from opening a named resource, and ssh carries it as its own @@ -19,7 +19,7 @@ channel type precisely because the two capabilities must not share one grant. The capability is dangerous in exactly the way ADR-001's "resource, not -address" framing protects against — arbitrary egress from the +address" framing protects against: arbitrary egress from the producer's network. The mitigation is not refusing the capability (a SOCKS5 service IS the capability) but separating it at the ACL: "dial `postgres-primary`" and "dial anything" must not be the same grant. @@ -27,7 +27,7 @@ The alksocks POC #5 scope gate (`socks5:bind`) proved the pattern matters in practice. Timing: alksocks is the first real consumer and nothing is published -against the direct shape yet — this is the correct moment to pin the +against the direct shape yet, so this is the correct moment to pin the wire-stable surface. ## Decision @@ -35,19 +35,19 @@ wire-stable surface. ### 1. One ALPN, a second open op The direct op rides the same `alk/tunnel` ALPN (ADR-002 stands: one -ALPN, one protocol identity — SSH's one-transport-many-channel-types -shape). The op id is **`channels/tunnel/direct`** — the flavor-form +ALPN, one protocol identity, i.e. SSH's one-transport-many-channel-types +shape). The op id is **`channels/tunnel/direct`**, the flavor-form op-id convention (ADR-002 Amendment 1): `channels//`, `Sub`-typed, same establishment phase and wrapper as `…/sub`. An older -producer rejects the unknown op loudly (registry lookup fails — the +producer rejects the unknown op loudly (registry lookup fails, which is the SSH "unknown channel type" posture, `NOT_FOUND`). The op-id convention itself needed an upstream alkcall amendment (ADR-047 amendment: discovery's ALPN derivation and the `rebuild_spec_for` marker path recognize flavor-form op ids; hub relay -reconstructs them). Runtime was never blocked — registration sets the +reconstructs them). Runtime was never blocked, since registration sets the ALPN explicitly and `open_channel(op_id, params, alpn)` takes the ALPN -as an argument — but discovery + hub relay are the re-produce path +as an argument; discovery + hub relay are the re-produce path (alknodes, OQ-TN-12), so the convention is pinned upstream before the first consumer. **Landed in alkcall 0.8.0** (review 008 U-1, ADR-047 amendment 3). @@ -61,27 +61,27 @@ amendment 3). } ``` -- **`substrate`** (string, required) — same discriminator semantics as +- **`substrate`** (string, required): same discriminator semantics as ADR-001: `"tcp"` | `"udp"` | `"unix"` in v1; selects the data-plane framing (ADR-003). Enum-validated; unknown values fail schema validation loudly. -- **`target`** (object, required) — a self-describing JSON object keyed +- **`target`** (object, required): a self-describing JSON object keyed by the sibling `substrate` field: - - `tcp` → `{ "host": , "port": }` — `host` is a + - `tcp` → `{ "host": , "port": }`, where `host` is a hostname, IPv4/IPv6 literal, or the JSON form of `SocketAddr`-shaped values; **domain-form targets stay unresolved - on the wire** (producer-side resolution — the ssh `-D` semantic; + on the wire** (producer-side resolution, the ssh `-D` semantic; both POCs validated egress-side resolution). No DNS dependency on the consumer. - - `udp` → `{ "host": , "port": }` — the connected-UDP + - `udp` → `{ "host": , "port": }`: the connected-UDP destination (endpoint-at-open; ADR-003 framing applies as for base udp). - - `unix` → `{ "path": }` — the socket path on the + - `unix` → `{ "path": }`: the socket path on the producer's filesystem. - The schema pins both fields required; the target sub-schema is per-substrate (an `allOf` of two `if`/`then` branches keyed on the - sibling `substrate` value, validated **registry-side** — the schema - gate runs before the establisher; the establisher's serde parse + sibling `substrate` value, validated **registry-side**: the schema + gate runs before the establisher; the establisher's serde parse re-checks as the backstop; see the Erratum below and Consequences). Malformed targets are the typed `invalid_input`-class failure; **`unknown_resource` can never fire on the direct op** (there is no @@ -97,16 +97,16 @@ amendment 3). *wire* encodes. - **Dial-only: no listen/direct-BIND form.** The reverse shape (`-R`-style ingress) is the listen-tunnel + the forwarded op - (ADR-008) — a direct-BIND form would duplicate its capability with + (ADR-008); a direct-BIND form would duplicate its capability with none of its metadata fidelity. ### 3. Scope: `TUNNEL_DIRECT_SCOPE = "tunnel:direct"` -The op's `AccessControl.required_scopes` is `["tunnel:direct"]` — a +The op's `AccessControl.required_scopes` is `["tunnel:direct"]`, a separate grant from `tunnel:open`, never implied by it. "Dial `postgres-primary`" and "dial anything from my network" are different capabilities; the ACL separation is structural (scopes attach to op -specs, not params — ADR-006 §2's additivity rule). Scope strings are +specs, not params, per ADR-006 §2's additivity rule). Scope strings are ACL-stable once published (ADR-006 §2): renames are breaking; new gates are new scopes on new ops. @@ -115,13 +115,13 @@ gates are new scopes on new ops. Same establisher contract as the dial shape (producer.md), minus the lookup: -1. Parse and validate `target` against `substrate` (a mismatch — - `unix` without `path`, `tcp` with a bad port — is `invalid_input` +1. Parse and validate `target` against `substrate` (a mismatch, such as + `unix` without `path` or `tcp` with a bad port, is `invalid_input` class). 2. Inject the parsed `TunnelTarget` into the assembly-provided `DialFn` (the same halves function the base op's registry backings - resolve to; ADR-004 — the protocol never sees a socket type). -3. Return the dialed handle via `Establishment::new(plan)` — unchanged + resolve to; ADR-004, which is why the protocol never sees a socket type). +3. Return the dialed handle via `Establishment::new(plan)`: unchanged plan-flow. Error mapping (the dial table, minus the registry row): @@ -163,12 +163,12 @@ sentence (also ADR-006 amendment). a target object must be optional-with-default. - **`SubstrateAddr`'s untagged leniency is schema-gated** (review 002 N-4): untagged serde variants cannot `deny_unknown_fields`, so a - cross-shape target (e.g. `{path, host}`) parses at the serde layer — - the SCHEMA's `additionalProperties: false` per branch is the real + cross-shape target (e.g. `{path, host}`) parses at the serde layer; + the SCHEMA's `additionalProperties: false` per branch is the real shape-vs-substrate gate. One caution for future registry variants: a schema-less registry would leave the establisher's serde backstop as the only gate, and it keys `render_target` off the sibling - `substrate` — a cross-shape target that sneaked past would render + `substrate`; a cross-shape target that sneaked past would render *something* (`/p` for the unix case) rather than fail; the serde backstop renders the sibling-keyed target, so cross-shape garbage is the schema's absence, not a serde failure. If this crate is ever @@ -177,15 +177,15 @@ sentence (also ADR-006 amendment). - **`SubstrateAddr` must not grow overlapping untagged variants.** If a future variant's shapes overlap (e.g. a `{host, port, flowlabel}` form), untagged disambiguation starts ordering (first-match wins - becomes load-bearing and silent) — switch to an internally-tagged - or untagged-with-content form at that point, deliberately (an ADR - line), never by accident. Pre-consumer and additive until then. -- **The BAST question is moot** — params is JSON inside the channels + becomes load-bearing and silent), which means switching to an internally-tagged + or untagged-with-content form at that point, deliberately (an ADR + line), never by accident. Pre-consumer and additive until then. +- **The BAST question is moot**: params is JSON inside the channels open op; the data plane is unchanged (raw pass-through for tcp/unix, - the `[len: u16 BE]` codec for udp — ADR-003). + the `[len: u16 BE]` codec for udp per ADR-003). - **Discovery exposure**: the direct op's `input_schema` carries `{substrate, target}`, so a discovery listing reveals that dynamic - egress exists and of which substrates — acceptable; the target + egress exists and of which substrates; that is acceptable, since the target values (what was dialed) never enter discovery (they ride per-call params). A producer wanting to hide the capability entirely simply does not register the op (and old producers reject it loudly anyway). @@ -196,7 +196,7 @@ sentence (also ADR-006 amendment). direct op itself is protocol-layer (default crate, wasm-clean). - **SOCKS5 CONNECT composes directly:** the alksocks adapter's `DialFn` seam maps the SOCKS5 CONNECT target (ATYP-decoded) onto - `open_direct` — the graduation's §4 table, now with the stand-in + `open_direct`, which is the graduation's §4 table with the stand-in dissolved. ## Erratum (2026-09-20, review 002 D-2) @@ -204,7 +204,7 @@ sentence (also ADR-006 amendment). §2's original wording said the target sub-schema is "`oneOf` on the sibling `substrate` value, validated at establisher time". Both halves of that sentence were stale against the shipped schema -(the decision itself was not — the code is the wire-stable side and +(the decision itself was not; the code is the wire-stable side and is correct): 1. **The shipped structure is an `allOf` of two `if`/`then` @@ -225,14 +225,14 @@ is correct): ## References - `docs/research/tunnels-graduation.md` §2 (the ask, options, and the - lean) — the design discussion this ADR records + lean): the design discussion this ADR records - alksocks `poc-bind-findings.md` (the scope-gate precedent) and `poc-connect-wasm-findings.md` (the dial-at-command-read layering) - alkcall ADR-047 (open ops + `input_schema` + discovery marker), ADR-049 (establishment/typed errors), Gap F (op-name → ALPN - derivation — the flavor-form amendment target) + derivation, the flavor-form amendment target) - alktunnels ADR-001 (the resource-params invariant the direct op extends), ADR-002 Amendment 1 (flavor-form op ids), ADR-006 (scope posture + the hub-policy amendment) -- AGENTS.md conventions 9 (no forced binding — dial-only, no listen +- AGENTS.md conventions 9 (no forced binding; dial-only, no listen form), 13 (access control), 17 (ALPN naming) \ No newline at end of file diff --git a/docs/architecture/decisions/008-listen-metadata-forwarded-op.md b/docs/architecture/decisions/008-listen-metadata-forwarded-op.md index 01285b0..9466436 100644 --- a/docs/architecture/decisions/008-listen-metadata-forwarded-op.md +++ b/docs/architecture/decisions/008-listen-metadata-forwarded-op.md @@ -1,9 +1,9 @@ -# ADR-008: Listen Metadata — Bind-First Establisher and the Forwarded Op +# ADR-008: Listen Metadata: Bind-First Establisher and the Forwarded Op ## Status -Accepted (2026-09-16); amended 2026-09-18 (the correlation token — -`listen` reply field + forwarded-params key, see Amendment 1); amended +Accepted (2026-09-16); amended 2026-09-18 (the correlation token, +the `listen` reply field + forwarded-params key, see Amendment 1); amended 2026-11-03 (the forwarded op's serving-side establishment phase + the correlation-miss vocabulary, see Amendment 2) @@ -12,12 +12,12 @@ correlation-miss vocabulary, see Amendment 2) alksocks' BIND POC (`poc-bind-findings.md` in the alksocks repo) pinned two upstream asks for composed (`-R`-style) BIND fidelity: -1. **Bound-addr discovery without consuming an accept** — reply#1's +1. **Bound-addr discovery without consuming an accept**: reply#1's `BND.ADDR` must precede the app server's dial, but the listen establisher is accept-first: its establishment phase pops an already-accepted connection, so the open cannot complete (and cannot report a bound address) until the app server dials. -2. **Peer metadata on accepted handles** — reply#2's peer address is +2. **Peer metadata on accepted handles**: reply#2's peer address is lost through the composed path; `TargetHandle` carries boxed halves only, so the composed variant reports the sentinel. @@ -25,8 +25,8 @@ The graduation research (`docs/research/tunnels-graduation.md` §3) framed the option space: per-ask carriers (a reply-metadata projection plus one of three addr-discovery shapes) versus ssh's `forwarded-tcpip` shape as the combined alternative. Decision: the -`forwarded-tcpip` shape — **bind-first establisher + accept-as-open** -— because it resolves both asks *and* two structural problems the POC +`forwarded-tcpip` shape (**bind-first establisher + accept-as-open**), +because it resolves both asks *and* two structural problems the POC recorded, with zero special carriers: - The address-first-vs-accept-first **deadlock dissolves** instead of @@ -35,9 +35,9 @@ recorded, with zero special carriers: OS-chosen bound address (ask #1 solved natively). - The **10s establishment-bound problem dissolves**: the accept wait moves out of the establishment phase into the pump-handler phase - (whose lifetime is teardown-governed, not establishment-bounded) — - FTP-style long BIND waits stop resolving as spurious refusals. -- Ask #2 (peer metadata) rides the reverse channel's open-op params — + (whose lifetime is teardown-governed, not establishment-bounded), so + FTP-style long BIND waits stop resolving as spurious refusals. +- Ask #2 (peer metadata) rides the reverse channel's open-op params: the wire's native "this connection is for X" moment. All the machinery is POC-validated: `open_reverse_channel` + @@ -50,18 +50,18 @@ over a serving connection (the reverse POC's hub shape). ### 1. Third establisher shape: bind-first The listen establisher gains a bind-first variant alongside the -accept-first shape (which stays — see §5): +accept-first shape (which stays; see §5): - The establisher **binds the listener** (via the assembly-provided listen function) and returns `Establishment::new(plan)` carrying the - listener handle. Establishment is fast — bounded by the bind, not by + listener handle. Establishment is fast, bounded by the bind, not by any app-server dial. - **The establisher mints a correlation token** (Amendment 1): a fresh - u32 unique per open on this producer, carried twice — inside the + u32 unique per open on this producer, carried twice: inside the plan (the pump handler needs it to correlate per-accept forwarded opens) and as the additive `listen` reply field (§2). The plan also carries the listen tunnel's substrate, stamped at establisher time - from the one successful params parse — the accept dispatcher reads + from the one successful params parse; the accept dispatcher reads it from the plan and stamps it into each forwarded open's `substrate` param; it never re-parses wire input (review 002 M-1: the removed silent `Tcp` fallback). The original @@ -71,24 +71,25 @@ accept-first shape (which stays — see §5): a generic hub relay forwards params untouched (it strips/rewrites only the reply's `channel_id`), so a spoke's forwarded open would carry an id the end opener never allocated. A producer-minted token - rides both hops untouched — reply fields pass through relays + rides both hops untouched, since reply fields pass through relays per-hop-truthful (alkcall ADR-051), and the same minted token appears in every hop's reply. - The establisher MAY contribute the observed bound address to the - open-op reply (§2): `"bound": { ... }` — the real OS-chosen address + open-op reply (§2): `"bound": { ... }`, which is the real OS-chosen address (port `0` resolved), substrate-shaped like the direct op's target (`{host, port}` for tcp/udp, `{path}` for unix). - The pump handler for this shape is an **accept dispatcher**, not `pump_bidi`: it accepts the channel's `BiStream` (the R-02 contract) - and runs the assembly-provided accept loop concurrently — each - accepted connection produces a forwarded channel (§3); the channel's - own `BiStream` stays idle and its EOF (the opener dropping the - listener session) cancels the loop and tears the listener down. The - idle-read check is deliberately WIDER than a bare EOF comparison - (review 002 N-2): the watch treats ANY read as teardown — `Ok(0)` - (the EOF sentinel), an error, **and** any non-empty read (a peer - that writes to the idle listen channel is misbehaving; the channel - is not a data plane — fail closed). An implementer reading only + and runs the assembly-provided accept loop concurrently; each + accepted connection produces a forwarded channel (§3); the channel's + own `BiStream` stays idle and its EOF (the opener dropping the + listener session) cancels the loop and tears the listener down. The + idle-read check is deliberately WIDER than a bare EOF comparison + (review 002 N-2): the watch treats ANY read as teardown, whether + `Ok(0)` + (the EOF sentinel), an error, **or** any non-empty read (a peer + that writes to the idle listen channel is misbehaving; the channel + is not a data plane, so fail closed). An implementer reading only "its EOF cancels the loop" and checking `Ok(0)` alone would hang on a write-happy peer; the shipped posture is the any-byte one, pinned by the write-happy-peer test. @@ -102,42 +103,42 @@ The open-op reply gains an **optional, additive** field: ``` - Present only when the establisher supplies it (bind-first listeners); - absent for dial establishers — old consumers ignore unknown reply + absent for dial establishers; old consumers ignore unknown reply fields (plain JSON, alkcall `ResponseEnvelope`), so the change is wire-additive. - The carrier is an upstream alkcall ask (ADR-049 amendment 3: - establisher → reply projection — `Establishment` gains optional + establisher → reply projection, with `Establishment` gaining optional reply fields the wrapper merges into the success output; the `channel_id` key is wrapper-reserved). Filed in `alkcall/docs/reviews/008-tunnels-graduation-upstream-review.md`. **Landed in alkcall 0.8.0** (review 008 U-2, ADR-049 amendment 3: `Establishment::with_reply_field` + the wrapper merge with the reserved-key guard). -- **`listen` — the correlation token (Amendment 1).** A bind-first +- **`listen`: the correlation token (Amendment 1).** A bind-first establisher also contributes `"listen": `: the token the forwarded op's params key on (§3, as amended). Same projection - carrier, same additive posture — a second reply field on the same - opens that carry `bound`. `bound` and `listen` are independent - fields (an accept-first establisher could supply `listen`-shaped - correlation too — the accept queue's establisher mints its own — - but v1 pins the pair on bind-first only). + carrier, same additive posture: a second reply field on the + same opens that carry `bound`. `bound` and `listen` are independent + fields (an accept-first establisher could supply `listen`-shaped + correlation too, since the accept queue's establisher mints its + own, but v1 pins the pair on bind-first only). - `bound` is informational surface, not addressing: a consumer MUST - NOT treat it as a dial target (the producer owns the backing — - ADR-001's invariant). It exists for RFC-1928-reply-class fidelity + NOT treat it as a dial target (the producer owns the backing, + per ADR-001's invariant). It exists for RFC-1928-reply-class fidelity (BND.ADDR) and observability. ### 3. The forwarded op: `channels/tunnel/forwarded` When the listener accepts, the **producer opens a channel back toward -the listen-opener** — accept-as-open, the `forwarded-tcpip` shape: +the listen-opener**: accept-as-open, the `forwarded-tcpip` shape: - Op id **`channels/tunnel/forwarded`** (flavor-form, ADR-002 Amendment 1), `Sub`-typed, ALPN marker `alk/tunnel`. It is served by the **listen-opener side** (the initiator of the listen-tunnel open): the producer *calls* it per accept - (`open_reverse_channel`-shaped — the call plane on channel 0), the + (`open_reverse_channel`-shaped; the call plane on channel 0), the opener's registry allocates the `channel_id`, the producer adopts it - (the ADR-047 §5 connection-owner rule — the serving side of the + (the ADR-047 §5 connection-owner rule, i.e. the serving side of the forwarded call allocates; the producer-side initiator adopts) and pumps the accepted connection's halves against the adopted channel. - Params: @@ -150,19 +151,19 @@ the listen-opener** — accept-as-open, the `forwarded-tcpip` shape: } ``` - - **`listen`** (u32, required) — the correlation token the + - **`listen`** (u32, required): the correlation token the bind-first establisher minted and returned in the listen open's reply (§2, as amended). The opener matches it against the - `listen` field of its open listener sessions — a `HashMap` - on the opener side; no registry op, no extra wire field. The - token space is the producer's: fresh per open, unique on that - producer (the opener-side map is keyed by the producer's token, - and one opener's map may hold tokens from several producers — a - hash collision across producers is the opener's bookkeeping + `listen` field of its open listener sessions, a `HashMap` + on the opener side; no registry op, no extra wire field. The + token space is the producer's: fresh per open, unique on that + producer (the opener-side map is keyed by the producer's token, + and one opener's map may hold tokens from several producers; a + hash collision across producers is the opener's bookkeeping choice, e.g. keying `(token)` uniquely or `(producer, token)`; v1 leaves it assembly-side). **Amends the original `channel` - key** (the listen channel's ID): that semantic is unimplementable - as pinned — the serving-side wrapper never exposes the allocated + key** (the listen channel's ID): that semantic is unimplementable + as pinned, since the serving-side wrapper never exposes the allocated channel ID to the establisher or handler (alkcall 0.8.0's `OpenHandler` surface), and generic hub relays forward params untouched, so a spoke's forwarded open would carry an id the end @@ -171,34 +172,35 @@ the listen-opener** — accept-as-open, the `forwarded-tcpip` shape: the reply field passes through per-hop truthful (alkcall ADR-051), and the forwarded open carries the same minted value at every hop. - - **`substrate`** (string, required) — the listen tunnel's + - **`substrate`** (string, required): the listen tunnel's substrate; shapes the forwarded channel's data-plane framing (ADR-003), same discriminator semantics as everywhere else. - - **`peer`** (object, required) — the accepted connection's remote + - **`peer`** (object, required): the accepted connection's remote address, substrate-shaped (`{host, port}` / `{path}`). This is ask #2's carrier: reply#2-class fidelity rides the open op, the wire's - native metadata moment. Optional in kind, required in v1 — a + native metadata moment. Optional in kind, required in v1: a producer that cannot know the peer (rare; all POC'd listeners can) omits... no: **required**; a producer that cannot determine the peer does not serve a bind-first listener over that substrate. -- Scope: **`TUNNEL_FORWARDED_SCOPE = "tunnel:forwarded"`** — the +- Scope: **`TUNNEL_FORWARDED_SCOPE = "tunnel:forwarded"`**, the opener-side grant for RECEIVING forwarded connections (ingress). Separate from `tunnel:open` and `tunnel:direct` (ADR-006 amendment): exposing a listener and dialing egress are different capabilities. - The opener-side serving shape: `register_openable_with_establisher` - as-is (no establisher — the forwarded channel needs no - establishment; its data plane starts immediately — **amended by - Amendment 2**: the served shape gains an establisher variant, the - injected correlation gate; this sentence records the pre-amendment - posture, which stays a valid shape — see Amendment 2), ACL-gated by + as-is (no establisher, because the forwarded channel needs no + establishment; its data plane starts immediately; this clause was + **amended by Amendment 2**: the served shape gains an establisher + variant, the injected correlation gate; this sentence records the + pre-amendment posture, which stays a valid shape per Amendment 2), + ACL-gated by the scope, handler = correlate the open (via `listen`, the correlation token) to the waiting session, accept the `BiStream`, pump against the local BIND client's - halves. The correlation registry is assembly-side bookkeeping — + halves. The correlation registry is assembly-side bookkeeping; the protocol provides the op and the params, not the wait/quiet-state machine. - An older opener that does not serve the forwarded op rejects the - producer's per-accept call loudly (`NOT_FOUND`) — the SSH posture. + producer's per-accept call loudly (`NOT_FOUND`), the SSH posture. A producer serving bind-first SHOULD prefer openers that advertise the forwarded op (discovery, alkcall ADR-047). @@ -210,7 +212,7 @@ let bound = listener.bound(); // the resolved ad // ... per forwarded accept (opener side, served op): // correlate by `listen`, accept the BiStream, pump against the // local session's halves -listener.close(self); // or Drop — tears the listen channel + far listener down +listener.close(self); // or Drop; tears the listen channel + far listener down ``` `TunnelListener` is the listen-flow sibling of `TunnelSession`: it owns @@ -218,7 +220,7 @@ the listen channel's teardown (ADR-005's ownership posture extended to the listen flow); it presents `bound()` and the teardown API, and the forwarded channels themselves arrive as separate adopted-side `TunnelSession`s on the opener's manager. `TunnelSession::adopt` gains -nothing — forwarded channels are ordinary adopted channels. The +nothing; forwarded channels are ordinary adopted channels. The opener-side correlation registry (token → waiting listener session) is assembly-side bookkeeping (consumer.md's posture); the crate's `TunnelListener` type provides the `listen` token it needs to key it. @@ -230,7 +232,7 @@ remains a valid establisher shape for producers whose resource backing IS an accept queue (an externally-fed or pre-bound listener shared across opens). Bind-first is the shape for producers that own the bind decision (the common hub case; alksocks' composed BIND). The two -shapes are per-registration assembly choices over the same op — no +shapes are per-registration assembly choices over the same op; there is no wire distinction beyond `bound`'s presence. ## Consequences @@ -243,7 +245,7 @@ wire distinction beyond `bound`'s presence. (ADR-049 amendment 3) and the flavor-form discovery derivation (ADR-047 amendment) **landed in alkcall 0.8.0** (review 008, Units 1–2); the in-tree `ChannelRelay` + `HubLegTemplate` (ADR-051) came - with them — the hub re-produce machinery §6 assumed assembly-layer + with them, so the hub re-produce machinery §6 assumed assembly-layer is now upstream, and the relay passes reply fields through per-hop truthful (which is what makes the Amendment 1 token compose through hubs with zero tunnel-specific relay behavior). The bind-first @@ -253,21 +255,21 @@ wire distinction beyond `bound`'s presence. carrier from the graduation doc is superseded: with accept-as-open, the peer rides the forwarded op's params; no process-local metadata field, no sentinel. (If a future non-forwarded accept shape needs - metadata, revisit then — additive.) + metadata, revisit then, and the change is additive.) - **The BIND reply#2 fidelity is end-to-end:** the opener-side correlation writes reply#2 with the REAL peer from the forwarded op's params. reply#1's `BND.ADDR` is the `bound` reply field. Both POC stand-ins (carried `bind_addr`; sentinel `0.0.0.0:0`) retire. - **Hub re-produce:** a hub re-producing a listen resource binds its own listener (bind-first, its own `bound` + minted `listen` token) - and re-opens forwarded channels with the same token — the spoke's + and re-opens forwarded channels with the same token, so the spoke's forwarded opens carry the hub's token untouched (params are not rewritten by the generic relay), and the metadata is per-hop truthful, matching alkcall ADR-042's terminate-and-re-produce - posture. The end opener correlates on the hub's minted token — the + posture. The end opener correlates on the hub's minted token, the value it read from its own open's reply. - **Local-variant BIND needs none of this** (the listener and the - accept are in-process — POC #5 variant 1, unchanged); the asks and + accept are in-process; POC #5 variant 1, unchanged); the asks and this ADR are scoped to the composed variant. ## References @@ -278,18 +280,18 @@ wire distinction beyond `bound`'s presence. deadlock and establishment-bound findings this shape dissolves) - alkcall ADR-047 (open ops; connection-owner allocation; the flavor-form amendment), ADR-049 (establishment; the reply-projection - amendment), ADR-042 (hub relay — re-produce id semantics) + amendment), ADR-042 (hub relay; re-produce id semantics) - alktunnels ADR-001 (params per-op amendment), ADR-002 Amendment 1 - (flavor-form op ids), ADR-005 (teardown ownership — extended to + (flavor-form op ids), ADR-005 (teardown ownership, extended to `TunnelListener`), ADR-006 (scope posture amendment) -- AGENTS.md conventions 9 (no forced binding — the bind decision is - the producer's), 13 (access control), 8 (two-pump contract — the +- AGENTS.md conventions 9 (no forced binding, so the bind decision is + the producer's), 13 (access control), 8 (two-pump contract, where the accept dispatcher is a per-accept `pump_bidi`, not a hand-rolled loop over one stream) ## Amendment 1 (the correlation token, 2026-09-18) -The decision stands — bind-first establisher, accept-as-open, the +The decision stands: bind-first establisher, accept-as-open, the `bound` reply field, the forwarded op. What changes: the forwarded op's correlation key (§3's `channel` param) and the bind-first establisher's reply surface (§2 gains `listen`). @@ -301,18 +303,18 @@ establisher's reply surface (§2 gains `listen`). 1. **The serving side never learns its own channel ID.** The open-op wrapper allocates the channel and replies with its ID, but the `OpenHandler`/establisher surface is - `Fn(Value, Option, Connection, AuthContext)` — no - ID anywhere. The §3 semantic ("the opener knows … without a + `Fn(Value, Option, Connection, AuthContext)`, with no + ID anywhere. The §3 semantic ("the opener knows … without a registry op") required the producer's accept dispatcher to know its own listen channel's ID to stamp into forwarded opens; - unimplementable without a new upstream surface. (Filing one — - exposing the allocated ID to handlers — was considered and + unimplementable without a new upstream surface. (Filing one, + which would expose the allocated ID to handlers, was considered and rejected: it is an alkcall-surface change for a correlation need this crate can solve in-protocol, and it would still not fix (2).) 2. **Generic hub relays do not rewrite params.** The in-tree - `ChannelRelay` (ADR-051) forwards the open-op input untouched — - it strips/rewrites only the reply's `channel_id`. A spoke's + `ChannelRelay` (ADR-051) forwards the open-op input untouched; + it strips/rewrites only the reply's `channel_id`. A spoke's forwarded opens would carry the spoke-side listen channel's ID, which the end opener never allocated and cannot correlate ("hubs re-map the id naturally" assumed tunnel-specific relay behavior @@ -323,23 +325,23 @@ establisher's reply surface (§2 gains `listen`). reply field via the same `Establishment::with_reply_field` carrier `bound` rides. Both fields pass through relays untouched (reply fields ride per-hop-truthful, ADR-051), and the forwarded - open's `listen` param carries the minted token at every hop — the + open's `listen` param carries the minted token at every hop; the end opener correlates on the value from its own open's reply. - **Why not a consumer-minted token:** the opener choosing the token and passing it in the listen open's params would work on the direct path, but fails hub re-produce the same way (the hub's own - bind-first open is a NEW open — it cannot adopt the end opener's + bind-first open is a NEW open, and it cannot adopt the end opener's token without a pass-through mechanism, which is exactly the params-rewriting relays don't do). Producer-minted is the direction information already flows (producer → opener via reply fields); no new flow is needed. -- **Token lifecycle:** minted at establisher time (one per open — +- **Token lifecycle:** minted at establisher time (one per open; two opens never share a token, so the opener's correlation map needs no session disambiguation beyond the token); retired with the listener session (the token's only consumer is the per-accept forwarded opens of that listener). Uniqueness is per-producer; the opener-side map keys on the token as read from its own reply. -- **Scope names, op ids, `bound`, `peer` — unchanged.** The wire +- **Scope names, op ids, `bound`, and `peer` are unchanged.** The wire delta vs the pre-amendment ADR is exactly: the reply gains the optional `listen` field, the forwarded params rename `channel` → `listen` with token semantics. Nothing else shipped against either @@ -349,17 +351,17 @@ establisher's reply surface (§2 gains `listen`). Resolves review 003 (`docs/reviews/003-correlation-miss-arrival-class-alkssh-input.md`) via `docs/plans/003-correlation-miss-resolve.md` §Decisions 1–4. The -decision stands — bind-first establisher, accept-as-open, the +decision stands: bind-first establisher, accept-as-open, the `bound`/`listen` reply fields, the forwarded op, `tunnel:forwarded`. What changes: **the serving side of the forwarded op (§3) gains an -establishment phase** — §3's sentence "no establisher — the forwarded +establishment phase**: §3's sentence, "no establisher; the forwarded channel needs no establishment; its data plane starts immediately" (marked inline above) is the text this amendment amends. ### The miss class this closes (the tri-vocabulary history) -A forwarded open whose `listen` token no longer routes — the -stale/missing-token arrival class — is currently served +A forwarded open whose `listen` token no longer routes, i.e. the +stale/missing-token arrival class, is currently served establisher-less everywhere (alkssh's door, alksocks' composed BIND, and this crate's built-in shape). That is the one posture exception to ADR-049's "a refused open is a typed call error, never a phantom @@ -367,16 +369,16 @@ channel": the open succeeds at the call plane, declines post-open in the assembly, and the producer-side pump EOFs with no signal. The family has named the class three ways: -- **alkssh** — `correlation-miss` at `debug`: +- **alkssh** uses `correlation-miss` at `debug`: `observe::decline_arrival(row, "correlation-miss", listen)` + teardown-at-birth (`src/tunnel_map.rs`; tokens removed at session teardown by the shared per-fabric-leg `ListenRouter`). -- **alksocks** — prose at `warn`: "forwarded open with no waiting +- **alksocks** uses prose at `warn`: "forwarded open with no waiting session" (`src/composed/mod.rs:497`). -- **alktunnels** — the `handler_error`-class generic arm at `warn` +- **alktunnels** has the `handler_error`-class generic arm at `warn` (the bind-first accept loop's failed-open branch, `src/producer.rs:838-854`), which fires only for the - connection-gone/`NOT_FOUND`/ACL-refused subclass — an in-session + connection-gone/`NOT_FOUND`/ACL-refused subclass; an in-session miss (route removed while the fabric connection lives) succeeds at the call plane and logs nothing at all. @@ -384,7 +386,7 @@ Amendment 2 retires the prose variant and the fabric-side silence. ### The decisions -1. **The serving side gains an establisher variant — the gate is +1. **The serving side gains an establisher variant, and the gate is injected.** The protocol crate cannot hold the assembly's correlation map, so the establisher's check is an assembly-provided gate closure over the forwarded params @@ -392,38 +394,39 @@ Amendment 2 retires the prose variant and the fabric-side silence. alkcall `OpenEstablisher` dance and registers via `register_openable_with_establisher` (alkcall 0.10.x, verified present). Assemblies must NOT hand-roll the alkcall establisher - plumbing — the crate-owned registration surface is the only + plumbing; the crate-owned registration surface is the only supported path. The gate is a lookup, not a dial: establishment stays fast, bounded by the usual establisher rules only. 2. **A stale/missing token fails the open as a typed `channel:open_failed`.** The crate maps the gate's failure to alkcall's `EstablishmentError::UnknownResource`, so the reply is - `channel:open_failed` with `details.reason = "unknown_resource"` — - no phantom channel is ever born (ADR-049's posture, extended here + `channel:open_failed` with `details.reason = "unknown_resource"`, + and no phantom channel is ever born (ADR-049's posture, extended here to the forwarded op's serving side). The waiting session IS a resource that is gone; **`unknown_resource` is reused as-is and the wire vocabulary is NOT extended**: `wire.md`'s open-op sections (the pinned `details.reason ∈ dial_failed / unknown_resource / resource_shortage / handler_error / timeout` list and its - §The Forwarded Op) are untouched by this amendment — there is no + §The Forwarded Op) are untouched by this amendment; there is no reason-class ADR, no new wire value. (The one serving-shape prose - line in `wire.md` — "no establisher" under §The Forwarded Op — + line in `wire.md`, "no establisher" under §The Forwarded Op, syncs with the API task's in-code doc updates, plan 003 W1; this ADR does not touch `wire.md`.) 3. **Gate order is pinned: the scope gate precedes the establisher.** The registry runs `tunnel:forwarded` ACL before the open op's establishment phase (alkcall's open-op wrapper order: registry - gates — schema, then ACL — then the wrapper's allocate → + gates (schema, then ACL), then the wrapper's allocate → establisher → handler). A scope-less producer is `FORBIDDEN` before any gate lookup; the gate never sees an unauthorized open. 4. **The residual post-open race keeps teardown-at-birth, named `correlation-miss` at `debug`.** A token that vanishes between - establisher pass and handler run — the narrow race the gate cannot - close — keeps §3's opener-half posture, unchanged: the assembly + establisher pass and handler run, the narrow race the gate cannot + close, keeps §3's opener-half posture, unchanged: the assembly handler declines, its early return reaps the channel teardown-at-birth (R-02). Its event name is **`correlation-miss`** - (alkssh's shipped vocabulary wins — it is published; alksocks' - prose variant retires), at `debug` **assembly-side** — the + (alkssh's shipped vocabulary wins; it is published, and alksocks' + prose variant retires), at `debug` **assembly-side**: + the protocol crate's built-in handler holds no map, so this is the assemblies' convention this ADR records, not code this crate runs. The producer-side generic arm (the accept loop's failed-open @@ -436,7 +439,7 @@ The base `register_tunnel_forwarded_openable` (establisher-less, the post-open shape) stays for compatibility; the establisher variant (`register_tunnel_forwarded_openable_with_establisher`) is the recommended posture, and new assemblies take it by default. Both ride -the same op id, params, scope, and reply surface — the wire delta of +the same op id, params, scope, and reply surface; the wire delta of this amendment is empty. ### References @@ -452,5 +455,5 @@ this amendment is empty. `EstablishmentError::UnknownResource → unknown_resource` (verified) - alkssh `src/tunnel_map.rs` (`ListenRouter`, `decline_arrival`); alksocks `src/composed/mod.rs:497`; alktunnels - `src/producer.rs:838-854` — the three miss sites this amendment + `src/producer.rs:838-854`: the three miss sites this amendment converges (source for §The miss class above) \ No newline at end of file diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md index f5dbaeb..29ce85f 100644 --- a/docs/architecture/open-questions.md +++ b/docs/architecture/open-questions.md @@ -1,9 +1,9 @@ --- status: draft -last_updated: 2026-09-18 (OQ-TN-16 updated — ADR-008 Amendment 1: the correlation token) +last_updated: 2026-09-18 (OQ-TN-16 updated per ADR-008 Amendment 1: the correlation token) --- -# alktunnels — Open Questions +# alktunnels: Open Questions Centralized tracker. The Phase 0 OQ ledger (`docs/research/ phase-0-findings.md`, OQ-TN-01..10) is promoted here with statuses @@ -15,7 +15,7 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. ### OQ-TN-01: Target addressing format (open-op `params` shape) - **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-01) -- **Status**: resolved — [ADR-001](decisions/001-open-params-layout.md) +- **Status**: resolved (see [ADR-001](decisions/001-open-params-layout.md)) - **Priority**: high - **Resolution**: 2026-09-07 (Phase 1). `params` = self-contained JSON object: `{ "resource": , "substrate": }`. @@ -26,17 +26,17 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. + survey input (ssh-socks5-survey §OQ-TN-01 residue) + both POCs. - **Cross-references**: OQ-TN-02, OQ-TN-03, ADR-001, ADR-002 -### OQ-TN-02: Datagram substrates (UDP) — boundary preservation +### OQ-TN-02: Datagram substrates (UDP), boundary preservation - **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-02) -- **Status**: resolved — [ADR-003](decisions/003-codec-and-udp-framing.md) +- **Status**: resolved; see [ADR-003](decisions/003-codec-and-udp-framing.md) - **Priority**: high - **Resolution**: split by path (2026-09-06 survey) + F-2 mandate (2026-09-07 reverse POC). Base open-op UDP resources: endpoint-at-open (one channel = one flow). Dynamic/`-D` UDP: per-datagram addressing inside the tunnel payload, assembly-layer. Boundary preservation: mandatory `[len: u16 BE]` per-datagram - framing — mandatory for correctness, not cosmetics (F-2; the full + framing; this is mandatory for correctness, not cosmetics (F-2; the full rationale and the executable pin live in ADR-003). Under the codec, `len=0` is a legal empty datagram (forward POC validated); EOF is the channels-level sentinel; the layers never collide. @@ -46,7 +46,7 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. - **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-03) - **Priority**: high -- **Status**: resolved — hub-owns-the-connection model (2026-09-05) + +- **Status**: resolved: the hub-owns-the-connection model (2026-09-05) + the reverse-flow POC (2026-09-07, `reverse-poc-summary.md`). No protocol-level direction: role follows the resource. `-L`/`-R` are the same open op with the entry point on different machines; @@ -60,37 +60,37 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. - **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-04) - **Priority**: medium -- **Status**: resolved — binding is always assembly-layer and +- **Status**: resolved: binding is always assembly-layer and optional, on either side; the protocol never binds. API surface: dial flows with no local bind (both POCs), listen flows where the binding is the producer's establisher (listen variant, ADR-004), - unbound/abstract flows (in-process pipes — the pump is generic over + unbound/abstract flows (in-process pipes; the pump is generic over boxed halves). The concrete produce/consume surface is spec'd in producer.md / consumer.md (ADR-005). - **Cross-references**: OQ-TN-03, ADR-004, ADR-005 ## Theme 2: Crate architecture -### OQ-TN-05: Backend inversion point — `TunnelBackend` trait? +### OQ-TN-05: Backend inversion point (`TunnelBackend` trait?) - **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-05) -- **Status**: resolved — [ADR-004](decisions/004-no-backend-trait.md) +- **Status**: resolved: [ADR-004](decisions/004-no-backend-trait.md) - **Priority**: medium - **Resolution**: 2026-09-07 (Phase 1). No trait for substrate access: "produce boxed halves for a resource" is a function (UDP POC, - OQ-TN-05 executable input). The only remaining thread — the hub - re-produce composition — resolves the same way: a hub proxy is a + OQ-TN-05 executable input). The only remaining thread, the hub + re-produce composition, resolves the same way: a hub proxy is a consumer + producer composed at the assembly layer (the reverse POC's hub is exactly this shape minus re-export); a composition helper is deferred(scope) until a concrete hub-proxy consumer exists (OQ-TN-12). - **Cross-references**: OQ-TN-12, ADR-004 -### OQ-TN-06: The two-pump helper — extract now? +### OQ-TN-06: The two-pump helper (extract now?) - **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-06) - **Priority**: medium -- **Status**: resolved (upstream) — `alkcall::channels::pump_bidi` +- **Status**: resolved (upstream): `alkcall::channels::pump_bidi` landed in alkcall 0.6.0 (ADR-050, review 007 R-03), shape as this OQ sketched it; this crate is cited as convergence input. alktunnels consumes the helper; hand-rolling is out. @@ -101,7 +101,7 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. ### OQ-TN-07: ALPN strategy - **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-07) -- **Status**: resolved — [ADR-002](decisions/002-alpn-strategy.md) +- **Status**: resolved: [ADR-002](decisions/002-alpn-strategy.md) - **Priority**: high - **Resolution**: 2026-09-07 (Phase 1). Option A: single `alk/tunnel` ALPN; the substrate discriminator in `params` selects the framing. @@ -113,8 +113,8 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. ### OQ-TN-08: Access control and ownership scope - **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-08) -- **Status**: resolved — [ADR-006](decisions/006-access-control-posture.md) - (2026-09-07 — the posture promoted to a decision). The resource is +- **Status**: resolved: [ADR-006](decisions/006-access-control-posture.md) + (2026-09-07; the posture promoted to a decision). The resource is owned by the far side (alkcall op-level ACL applies as-is; no tunnel-specific policy layer, no allowlists in v1; the ownership seam is the establisher seeing the per-call opener identity). @@ -134,7 +134,7 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. - **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-09) - **Priority**: high - **Status**: resolved (superseded by alkcall ADR-049, 0.5.0; narrowed - through 0.6.0/0.7.0) — establishment is the open op's awaited phase; + through 0.6.0/0.7.0): establishment is the open op's awaited phase; dial failure is a typed `channel:open_failed` call error (reason ∈ `dial_failed` / `unknown_resource` / `resource_shortage` / `handler_error` / `timeout`), never a phantom channel. Residual @@ -148,35 +148,35 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. - **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-10) - **Priority**: low -- **Status**: resolved — UDP POC DONE (2026-09-06, 17 tests), +- **Status**: resolved: the UDP POC is DONE (2026-09-06, 17 tests), reverse-flow POC DONE (2026-09-07, 14 tests; re-validated over alkcall 0.7.0, 16 tests). #3 split (OQ-TN-14): unix ships with the - `local` feature; stdio is OUT — alktty's pipe mode owns process + `local` feature; stdio is OUT, since alktty's pipe mode owns process stdio (exit codes + signals are tty semantics). #4 (helper spike) resolved by alkcall ADR-050. Findings F-1 (plan payloads - `Send + Sync` — documented upstream) and F-2 (UDP codec mandatory) + `Send + Sync`, documented upstream) and F-2 (UDP codec mandatory) recorded; W1 filed upstream and RESOLVED (CF-005/CF-006). - **Cross-references**: ADR-003, OQ-TN-05, OQ-TN-06 ## Theme 4: Phase 1 residues (new) -### OQ-TN-11: Resource naming — collision domain and lifecycle of a resource id +### OQ-TN-11: Resource naming (collision domain and lifecycle of a resource id) - **Origin**: OQ-TN-01's residue (stable resource name vs target address), carried into Phase 1 -- **Status**: partially resolved — the collision-domain half is +- **Status**: partially resolved: the collision-domain half is **resolved by [ADR-001](decisions/001-open-params-layout.md)**: the domain is the producer's registry (per-producer; two producers on - the same connection may expose the same resource name independently - — the open op resolves against the SERVING side's registry). The + the same connection may expose the same resource name independently; + the open op resolves against the SERVING side's registry). The lifecycle half stays open: is a resource registry a producer-side - assembly construct (the POCs' `HashMap` — the leading candidate), + assembly construct (the POCs' `HashMap`, the leading candidate), or does it need protocol-level register/rename/retire ops? - **Priority**: medium - **Impacts**: producer.md's `ResourceRegistry` shape and the discovery story (OQ-TN-08); a protocol-level registry op would be a new wire surface (one-way door). -- **Resolution**: lifecycle half pending — no prior-art pressure +- **Resolution**: lifecycle half pending; no prior-art pressure (SSH lists no forwards in-band; discovery is the ops listing); decide when a consumer needs shared/lifecycle-managed resources. - **Cross-references**: ADR-001, OQ-TN-08 @@ -184,7 +184,7 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. ### OQ-TN-12: Hub re-produce composition helper - **Origin**: OQ-TN-05's remaining thread (2026-09-05 hub model) -- **Status**: resolved (upstream, 2026-09-18) — alkcall 0.8.0 ships +- **Status**: resolved (upstream, 2026-09-18): alkcall 0.8.0 ships the in-tree `ChannelRelay` + `HubLegTemplate`/`HubLegImports` (alkcall ADR-051, review 008 Unit 3): the relay openable wraps from_call-imported marked specs on the consumer-leg registry via @@ -192,10 +192,10 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. teardown), the translate hop calls the producer leg with `forwarded_for` from the consumer's per-call identity, the byte-forward hop is `pump_bidi` awaited inline, the spoke reply's - `channel_id` is stripped/replaced (other fields ride through — + `channel_id` is stripped/replaced (other fields ride through, so `bound` flows end-to-end per-hop truthful), and the adopted producer-leg entry reclaims via a `Drop` guard on every plan path. - The composition helper a hub proxy needed is upstream — this crate + The composition helper a hub proxy needed is upstream, and this crate composes it (OQ-TN-05's resolution stands: no tunnel-side trait). Tracker task: `architecture/oq-tn-12-tracker` (tasks/architecture/). - **Priority**: low @@ -205,9 +205,9 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. ### OQ-TN-13: UDP truncation semantics at the codec boundary - **Origin**: forward POC Issues Surfaced #3 -- **Status**: resolved — [ADR-003](decisions/003-codec-and-udp-framing.md) +- **Status**: resolved: [ADR-003](decisions/003-codec-and-udp-framing.md) (2026-09-07): **fail-loud at the adapter level.** A receive buffer - smaller than the datagram is an error, never a silent truncation — + smaller than the datagram is an error, never a silent truncation; a truncated datagram would corrupt the length-framed stream's framing invariants (the reverse POC's F-2 rationale: the codec's wire discipline is what keeps empty datagrams unambiguous; silent @@ -226,20 +226,20 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. - **Origin**: OQ-TN-10 #3 (the cheap un-POC'd item) - **Priority**: low - **Status**: **resolved 2026-09-07 (split by substrate):** - - **Unix: in scope for the `local` feature v1** — same halves shape + - **Unix: in scope for the `local` feature v1**: same halves shape as TCP (`dial_unix`), cheap, wire enum already carries `unix` (ADR-001). Implemented by `tunnels/local-socket-halves`. - - **Stdio: OUT of scope — alktty owns it.** A spawned process's + - **Stdio: OUT of scope; alktty owns it.** A spawned process's stdin/stdout/stderr IS a terminal session in alktty's model: `LocalTtyBackend`'s pipe mode (`tokio::process::Command` + `Stdio::piped()`, alktty `tty-local.md`) is exactly the stdio - case — three multiplexed logical streams + the exit-code + case, with three multiplexed logical streams + the exit-code control chunk (alktty ADR-004) + signal forwarding (REQ-TTY-02). A "stdio bridge" here would be alktty's runner mode with the terminal stripped out: a strictly worse duplicate that also - drops the semantics that matter (exit code, signals — a byte + drops the semantics that matter (exit code, signals; a byte tunnel has neither). Consumers needing to run commands remotely - compose: alkcall dial → alktty session (pipe mode) — the two + compose: alkcall dial → alktty session (pipe mode); the two crates are siblings on the same channels substrate, not competitors. (The OQ-TN-10 #3 "cheap item" was scoped before the sibling relationship was this clear.) @@ -256,16 +256,16 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. - **Origin**: `docs/research/tunnels-graduation.md` §2 (the alksocks graduation discussion, 2026-09-15) - **Priority**: high -- **Status**: resolved — [ADR-007](decisions/007-direct-open-op.md) +- **Status**: resolved: [ADR-007](decisions/007-direct-open-op.md) (2026-09-16). Option (a): a second `Sub`-typed open op `channels/tunnel/direct` on the same `alk/tunnel` ALPN, params `{substrate, target}` (self-describing JSON target keyed by the - sibling substrate — ATYP bytes rejected: not schema-describable, no + sibling substrate; ATYP bytes rejected: not schema-describable, no unix path form), scope `tunnel:direct` (never implied by - `tunnel:open` — structural ACL separation), dial-only (no - listen/direct-BIND form — ingress is the listen + forwarded + `tunnel:open`, which is the structural ACL separation), dial-only (no + listen/direct-BIND form; ingress is the listen + forwarded surface). The op-id convention needed an upstream alkcall amendment - (ADR-047's discovery derivation was `/sub`-shaped) — filed in + (ADR-047's discovery derivation was `/sub`-shaped); it was filed in `alkcall/docs/reviews/008-tunnels-graduation-upstream-review.md` (ask #1) and **landed in alkcall 0.8.0** (ADR-047 amendment 3: flavor-form op ids + the explicit `channel_open_alpn` discovery @@ -279,23 +279,23 @@ OQ-TN numbering. Format per `docs/sdd_process.md`. BIND POC's two asks: bound-addr without consuming an accept; peer metadata on accepted handles) - **Priority**: high -- **Status**: resolved — [ADR-008](decisions/008-listen-metadata-forwarded-op.md) - (2026-09-16; **amended 2026-09-18** — Amendment 1). The +- **Status**: resolved: [ADR-008](decisions/008-listen-metadata-forwarded-op.md) + (2026-09-16; **amended 2026-09-18**, Amendment 1). The `forwarded-tcpip` shape adopted over per-ask carriers: (1) a - bind-first listen establisher — establishment ends at bind time, so + bind-first listen establisher: establishment ends at bind time, so the open-op reply carries the real OS-chosen address via an additive `bound` field (ask #1; the carrier is the establisher → - reply projection — filed as the review's ask #2 and **landed in + reply projection, filed as the review's ask #2 and **landed in alkcall 0.8.0** as ADR-049 amendment 3, `Establishment::with_reply_field` + `open_channel_with_reply`), and the address-first-vs-accept-first BIND deadlock + the 10s establishment-bound problem dissolve with the shape; (2) the - forwarded op `channels/tunnel/forwarded` (accept-as-open) — the + forwarded op `channels/tunnel/forwarded` (accept-as-open): the producer opens a channel per accept with `{listen, substrate, peer}` in params (ask #2's carrier; the `TargetHandle.peer` option is superseded). Amendment 1: the correlation key is a **producer-minted token** (`listen` reply field + forwarded - param), not the listen channel's ID — the serving-side handler + param), not the listen channel's ID; the serving-side handler never sees its own channel ID (alkcall 0.8.0's `OpenHandler` surface) and generic relays don't rewrite params, so a producer-minted token (which rides reply fields and params through diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index d9c2d6f..9527457 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -1,12 +1,12 @@ --- status: draft -last_updated: 2026-09-18 (the graduation: the direct op, listen metadata, the forwarded op — ADR-007/008 as amended: the correlation token) +last_updated: 2026-09-18 (the graduation: the direct op, listen metadata, the forwarded op; ADR-007/008 as amended: the correlation token) --- -# alktunnels — Overview +# alktunnels: Overview Arbitrary bidirectional tunnels over alkcall channels: TCP, UDP, unix -sockets, and other stream or datagram substrates — in the `ssh -L` / +sockets, and other stream or datagram substrates, all in the `ssh -L` / `ssh -D` / `ssh -R` sense. A producer/consumer protocol crate riding alkcall channels the same way alktty does (`alk/tty` is the sibling precedent). This document covers the crate's purpose, the resource @@ -28,8 +28,8 @@ channels and provides the typed consumer sessions: halves (ADR-050, R-02's lifetime contract). Three ops on the one ALPN (ADR-002 Amendment 1): `channels/tunnel/sub` (the resource op), `channels/tunnel/direct` (dynamic-target egress, ADR-007), and - — served by the listen-opener side — `channels/tunnel/forwarded` - (accept-as-open, ADR-008). Three establisher shapes: dial, listen + `channels/tunnel/forwarded` (served by the listen-opener side; + accept-as-open, ADR-008). Three establisher shapes: dial, listen (accept-first), and bind-first listen. - **The consumer half** (`TunnelSession` / `TunnelListener`, ADR-005/008): opens tunnel channels (`ChannelClient::open_channel` @@ -44,11 +44,11 @@ The guiding insight (the OQ-TN-01 reframe): > A tunnel is a resource, not an address. Base params identify a > produced resource + substrate; the producer owns the backing. The > one additive exception is the direct op (ADR-007): a consumer may -> ask the producer to dial an arbitrary target — a different -> capability, its own op and its own scope (`tunnel:direct`), never +> ask the producer to dial an arbitrary target, and that is a +> different capability, its own op and its own scope (`tunnel:direct`), never > implied by `tunnel:open`. Rich per-datagram addressing (SOCKS5 ATYP, > udpgw-style remotes) still enters only through the `-D`/dynamic -> composition path — inside the tunnel payload, never in the wire. +> composition path, i.e. inside the tunnel payload, never in the wire. ## Why @@ -57,15 +57,15 @@ consumers, the same role alktty plays for terminal sessions: - A hub that runs agent workspaces exposes workers' services (postgres, redis, gitea HTTP) as named resources; a coordinator - opens tunnels to them over alkcall channels — the VPN/SSH-tunnel - service pattern without SSH. + opens tunnels to them over alkcall channels (the VPN/SSH-tunnel + service pattern without SSH). - A worker behind NAT dials the hub and serves its own open ops (the reverse flow, `ssh -R`); the hub opens tunnel channels toward it per local accept. Both POCs validated this end-to-end. - A `-D`-style consumer tunnels a SOCKS5 connection; the socks5 server (assembly layer) does dynamic target selection inside the tunnel payload. -- The protocol layer is substrate-agnostic by construction — the +- The protocol layer is substrate-agnostic by construction: the bookkeeping (target addressing, direction, lifecycle) never hardcodes a substrate (AGENTS.md convention 7). TCP, UDP, unix sockets, and in-process pipes ride the same open op, pump shape, @@ -77,10 +77,10 @@ A tunnel channel's life (full detail in [wire.md](wire.md)): 1. **Open (the establishment phase).** The consumer calls `channels/tunnel/sub` with `params = {resource, substrate}` - (ADR-001) — or `channels/tunnel/direct` with + (ADR-001), or `channels/tunnel/direct` with `params = {substrate, target}` (ADR-007, dynamic egress). The - producer's establisher — the open op's awaited phase (alkcall - ADR-049) — resolves the resource (or parses the target), dials the + producer's establisher (the open op's awaited phase, alkcall + ADR-049) resolves the resource (or parses the target), dials the substrate, and returns the handle in the plan. Failure is a typed `channel:open_failed` (`dial_failed` / `unknown_resource` / `resource_shortage` / `handler_error` / `timeout`); the channel @@ -124,15 +124,15 @@ alktunnels (local feature) — non-wasm by design Same posture as alktty: the default crate is protocol-only and wasm-clean (AGENTS.md convention 4); substrate I/O is feature-gated (ADR-004). The wire format carries no alkcall-internal types -(alktty ADR-006's self-containment rule applies to the tunnel codec — -trivial here, since the codec is a 2-byte length prefix). +(alktty ADR-006's self-containment rule applies to the tunnel codec, +though it is trivial here, since the codec is a 2-byte length prefix). ## Feature Gates | Feature | Contents | Wasm | |---------|----------|------| -| *(default)* | params, wire codec, open-op spec, establisher shapes, `TunnelSession` — protocol only | yes | -| `local` | TCP/UDP/unix dial + listen helpers — halves-producing dial/listen functions (ADR-004). Unix ships (OQ-TN-14); stdio does NOT — a spawned process's stdio is alktty's pipe mode (exit codes, signals), not a tunnel | no | +| *(default)* | params, wire codec, open-op spec, establisher shapes, `TunnelSession` (protocol only) | yes | +| `local` | TCP/UDP/unix dial + listen helpers, i.e. halves-producing dial/listen functions (ADR-004). Unix ships (OQ-TN-14); stdio does NOT: a spawned process's stdio is alktty's pipe mode (exit codes, signals), not a tunnel | no | ## Module Map @@ -164,14 +164,14 @@ All design decisions are documented as ADRs in [decisions/](decisions/). | ADR | Decision | Summary | |-----|----------|---------| -| [001](decisions/001-open-params-layout.md) | Open-op params: resource + substrate | `{resource, substrate}` — the producer's stable name, not an address; wire-stable from the first consumer | +| [001](decisions/001-open-params-layout.md) | Open-op params: resource + substrate | `{resource, substrate}`, the producer's stable name (not an address); wire-stable from the first consumer | | [002](decisions/002-alpn-strategy.md) | Single `alk/tunnel` ALPN | Substrate in params selects the framing; no ALPN split, no API bifurcation | -| [003](decisions/003-codec-and-udp-framing.md) | Data-plane codec | Raw pass-through (stream) / mandatory `[len: u16 BE]` (UDP — F-2 mandate); `len=0` is a legal empty datagram | +| [003](decisions/003-codec-and-udp-framing.md) | Data-plane codec | Raw pass-through (stream) / mandatory `[len: u16 BE]` (UDP; F-2 mandate); `len=0` is a legal empty datagram | | [004](decisions/004-no-backend-trait.md) | No TunnelBackend trait | Halves functions at the assembly layer; listen is an establisher shape; trait re-evaluated at the 3-consumer threshold | -| [005](decisions/005-consumer-session-owns-teardown.md) | Consumer session owns teardown | `TunnelSession` with close/join/Drop — the W3 gap closes structurally | -| [006](decisions/006-access-control-posture.md) | Access-control posture | The open gate is the boundary: `tunnel:open` scope (stable once published), op-level ACL, ownership seam, no allowlists in v1; amended — `tunnel:direct`/`tunnel:forwarded` scopes + the hub re-produce policy sentence | -| [007](decisions/007-direct-open-op.md) | The direct open op | `channels/tunnel/direct`: dynamic-target egress, `{substrate, target}` params, `tunnel:direct` scope, dial-only — the ssh `direct-tcpip` capability with structural ACL separation | -| [008](decisions/008-listen-metadata-forwarded-op.md) | Listen metadata + the forwarded op | Bind-first establisher (reply `bound` field = BND.ADDR fidelity; `listen` = the correlation token), `channels/tunnel/forwarded` accept-as-open (params carry `{listen, substrate, peer}` = reply#2 fidelity) — the `forwarded-tcpip` shape; amended — producer-minted correlation token supersedes the channel-ID key | +| [005](decisions/005-consumer-session-owns-teardown.md) | Consumer session owns teardown | `TunnelSession` with close/join/Drop; the W3 gap closes structurally | +| [006](decisions/006-access-control-posture.md) | Access-control posture | The open gate is the boundary: `tunnel:open` scope (stable once published), op-level ACL, ownership seam, no allowlists in v1; amended with the `tunnel:direct`/`tunnel:forwarded` scopes + the hub re-produce policy sentence | +| [007](decisions/007-direct-open-op.md) | The direct open op | `channels/tunnel/direct`: dynamic-target egress, `{substrate, target}` params, `tunnel:direct` scope, dial-only (the ssh `direct-tcpip` capability with structural ACL separation) | +| [008](decisions/008-listen-metadata-forwarded-op.md) | Listen metadata + the forwarded op | Bind-first establisher (reply `bound` field = BND.ADDR fidelity; `listen` = the correlation token), `channels/tunnel/forwarded` accept-as-open (params carry `{listen, substrate, peer}` = reply#2 fidelity); this is the `forwarded-tcpip` shape; amended so that a producer-minted correlation token supersedes the channel-ID key | ## Open Questions @@ -179,13 +179,13 @@ Open questions are tracked in [open-questions.md](open-questions.md). Key questions affecting this document: - **OQ-TN-11**: resource naming collision domain + lifecycle (partially resolved) -- **OQ-TN-12**: hub re-produce composition helper (deferred(scope) — +- **OQ-TN-12**: hub re-produce composition helper (deferred(scope); the policy sentence is pinned: the hub's ACL governs the final dial) -- **OQ-TN-13**: UDP truncation semantics (resolved — fail-loud, ADR-003) -- **OQ-TN-14**: unix/stdio placement (resolved — unix in via +- **OQ-TN-13**: UDP truncation semantics (resolved, fail-loud, ADR-003) +- **OQ-TN-14**: unix/stdio placement (resolved; unix in via `local`, stdio out: alktty's pipe mode owns process stdio) -- **OQ-TN-15**: the direct op (resolved — ADR-007) -- **OQ-TN-16**: listen metadata (resolved — ADR-008) +- **OQ-TN-15**: the direct op (resolved; ADR-007) +- **OQ-TN-16**: listen metadata (resolved; ADR-008) ## References diff --git a/docs/architecture/producer.md b/docs/architecture/producer.md index 3c61ac5..9aff0e3 100644 --- a/docs/architecture/producer.md +++ b/docs/architecture/producer.md @@ -1,14 +1,14 @@ --- status: draft last_updated: 2026-11-03 (the forwarded op's serving-side establishment - phase + the correlation-miss vocabulary — ADR-008 Amendment 2) + phase + the correlation-miss vocabulary, ADR-008 Amendment 2) --- -# alktunnels — Producer Half +# alktunnels: Producer Half -The producer half: the `channels/tunnel/sub` open op — its spec, its +The producer half: the `channels/tunnel/sub` open op, its spec, its establishers (dial + listen shapes), its pump handler, and its -registration — plus the sibling ops on the same ALPN: the direct op +registration, plus the sibling ops on the same ALPN: the direct op (`channels/tunnel/direct`, dynamic-target egress) and the forwarded op (`channels/tunnel/forwarded`, accept-as-open). The producer is whoever can reach the tunnel target; connection direction is irrelevant @@ -44,16 +44,16 @@ ACL, allocates the channel (connection-owner rule, ADR-047 §5), runs the establisher as the awaited bounded establishment phase, then spawns the pump handler. The generic channel ops (`channel/close`, `channel/control`, `channel/resources/subscribe`) -register separately via `ChannelOperations::register_on` — the +register separately via `ChannelOperations::register_on`; the producer (or its assembly layer) MUST register them on the serving registry so peers can manage channels out-of-band (the recipe alkcall -review 007 §Part B verified — `alkcall/docs/reviews/ +review 007 §Part B verified; see `alkcall/docs/reviews/ 007-establishment-follow-ups-review.md`; R-01 = the `Establishment` plan payload, R-02 = the `OpenHandler` lifetime contract). **Registration timing:** the openable may register AFTER `from_connection_with_serving` returns (the connect-side serving -posture — the reverse POC's W2): the dispatcher reads through the +posture; the reverse POC's W2): the dispatcher reads through the shared `Arc` per dispatch. Late registration is a supported assembly shape, not a race. @@ -61,7 +61,7 @@ supported assembly shape, not a race. Dynamic-target egress (ADR-007): the same ALPN, a second `Sub`-typed open op whose establisher dials a per-call target instead of resolving -a resource. One ALPN, two ops, two scopes — dial-what-the-producer- +a resource. One ALPN, two ops, two scopes: dial-what-the-producer- registered (`tunnel:open`) and dial-anything (`tunnel:direct`) are different capabilities. @@ -77,11 +77,11 @@ access control: required_scopes: ["tunnel:direct"] description: SHOULD carry a human-readable hint (dynamic-target egress) ``` -- **No registry lookup** — the establisher parses `target` against +- **No registry lookup**: the establisher parses `target` against `substrate` (malformed → `invalid_input`-class; `unknown_resource` can never fire) and injects the parsed `TunnelTarget` into the same assembly-provided `DialFn` the base op's backings resolve to. -- **Dial-only** — no listen/direct-BIND form; the ingress capability +- **Dial-only**: no listen/direct-BIND form; the ingress capability is the listen + forwarded surface (ADR-008). - Registration: `register_tunnel_direct_openable` (same wrapper, same plan-flow; the spec builder is public for assembly variants). @@ -91,9 +91,9 @@ description: SHOULD carry a human-readable hint (dynamic-target egress) The establisher is the open op's awaited, bounded establishment phase (alkcall ADR-049 §1/§2): validate params semantically, consult ownership if wired, dial/accept the substrate, and return -`Establishment::new(plan)` — the dialed handle rides the typed-opaque +`Establishment::new(plan)`; the dialed handle rides the typed-opaque plan (`ChannelPlan = Arc`; the `+ Sync` bound -constrains plan payloads — socket-backed handles carry it naturally, +constrains plan payloads; socket-backed handles carry it naturally, reverse POC F-1, documented upstream). On failure: `EstablishmentError` (the typed reason codes) → the wrapper tears the channel down and replies `channel:open_failed`. @@ -126,15 +126,15 @@ listener; a hub exposing its own port to a third party): the same open op, the same params; the establisher pops the next accepted connection from an assembly-owned listener queue and returns it as the plan payload. The listener lifecycle (bind, accept loop, cancel) -is assembly code — the protocol never binds (OQ-TN-04). Error +is assembly code; the protocol never binds (OQ-TN-04). Error mapping: an empty queue during establishment is `resource_shortage`; -a closed listener is `dial_failed`-class. No new wire surface — the +a closed listener is `dial_failed`-class. No new wire surface: the listen variant is plan-flow with a different halves source. **The accept-first shape stays** (ADR-008 §5): for producers whose resource backing IS an accept queue (an externally-fed or pre-bound -listener shared across opens), this remains the correct shape — the -POC #5 composed-BIND shape. Its limits (no `bound` in the reply — the +listener shared across opens), this remains the correct shape: the +POC #5 composed-BIND shape. Its limits (no `bound` in the reply: the accept hasn't happened at establishment; the POC's establishment-bound-vs-wait problem) are inherent to accepting-first. @@ -142,32 +142,32 @@ establishment-bound-vs-wait problem) are inherent to accepting-first. For a producer that OWNS the bind decision (the common hub case; alksocks' composed BIND): the establisher **binds the listener** (via -the assembly-provided listen function — the protocol never binds, +the assembly-provided listen function; the protocol never binds, OQ-TN-04) and returns the listener handle as the plan payload. -Establishment is fast — bounded by the bind, not by any app-server -dial: +Establishment is fast (bounded by the bind, not by any app-server +dial): - **The establishment phase ends at bind time**, so the open-op reply can carry the REAL OS-chosen address: the establisher contributes - `"bound": {host, port}` (or `{path}`) to the reply — the + `"bound": {host, port}` (or `{path}`) to the reply: the establisher → reply projection (ADR-049 amendment 3, `Establishment::with_reply_field`; the `channel_id` key is wrapper-reserved). **Landed in alkcall 0.8.0** (review 008 U-2); consumers read it via `ChannelClient::open_channel_with_reply`. - **The establisher mints the correlation token** (ADR-008 Amendment 1): a fresh u32 per open, contributed as the `"listen"` - reply field (the same projection) and carried in the plan — the - accept dispatcher reads it from the plan and stamps it into each - forwarded open's `listen` param. The plan carries - `{listener, token, substrate}` (a bind-first plan type; the - substrate is stamped at establisher time — the dispatcher reads it - from the plan, never re-parses wire input); the token is - relay-transparent by construction (reply fields and params both - pass hubs untouched — alkcall ADR-051; ADR-008 Amendment 1 has the + reply field (the same projection) and carried in the plan; the + accept dispatcher reads it from the plan and stamps it into each + forwarded open's `listen` param. The plan carries + `{listener, token, substrate}` (a bind-first plan type; the + substrate is stamped at establisher time, and the dispatcher reads it + from the plan, never re-parses wire input); the token is + relay-transparent by construction (reply fields and params both + pass hubs untouched per alkcall ADR-051; ADR-008 Amendment 1 has the why-superseded of the channel-ID key). - **The pump handler for this shape is an accept dispatcher**, not `pump_bidi`: it accepts the channel's `BiStream` (the R-02 - contract), then runs the assembly-provided accept loop — each + contract), then runs the assembly-provided accept loop; each accepted connection produces a forwarded-channel open toward the listen opener (§The Forwarded Op below); the accepted halves pump against the adopted channel. The listen channel's own `BiStream` @@ -179,22 +179,22 @@ dial: - The address-first-vs-accept-first BIND deadlock (the alksocks POC's one real discovery) DISSOLVES in this shape: the open completes at bind time, reply#1-class metadata (the `bound` field) precedes any - app-server dial, and the accept wait lives in the pump phase — - teardown-governed, not establishment-bounded (the 10s bound problem + app-server dial, and the accept wait lives in the pump phase: + teardown-governed, not establishment-bounded (the 10s bound problem dissolves with it). **Identity:** the establisher receives the **per-call opener identity** (the dispatch-resolved identity overlaid onto the -install-time context — alkcall 0.7.0 CF-006). On hub-forwarded opens +install-time context, alkcall 0.7.0 CF-006). On hub-forwarded opens it sees the end client, not the hub. `auth` carries the opener's identity plus the transport-truthful fields (`alpn`, `remote_addr`, -`tls_client_fingerprint`) — never rewritten. Consult it for +`tls_client_fingerprint`); these are never rewritten. Consult it for ownership-scoped resources if the assembly layer wants `OwnershipProvider` checks here (OQ-TN-08's optional half). **Concurrency:** each open's establisher result flows to ITS pump -handler — concurrent opens of the same resource are independent -(the R-01 plan flow — alkcall review 007 R-01, the +handler; concurrent opens of the same resource are independent +(the R-01 plan flow per alkcall review 007 R-01, the `Establishment::new(plan)` payload; the forward POC's resource-keyed handoff race is structurally gone, confirmed under concurrency by the reverse POC). @@ -206,27 +206,27 @@ The `OpenHandler` (alkcall ADR-049/050 shape: JoinHandle<()>`): 1. Downcast the plan to the concrete handle type - (`Arc::downcast::` — establisher and handler agree + (`Arc::downcast::`, so establisher and handler agree on the type; alkcall never inspects it). -2. `conn.accept_bi()` — the channel's yield-once `BiStream`. -3. `pump_bidi(bidi, handle.read, handle.write)` — **awaited inline** +2. `conn.accept_bi()`: the channel's yield-once `BiStream`. +3. `pump_bidi(bidi, handle.read, handle.write)`: **awaited inline** inside the handler's task. The returned `JoinHandle` MUST track the data-plane lifetime (R-02): the wrapper awaits it and its completion triggers channel teardown. **Early return = - teardown-at-birth** — both pumps see instant EOF with zero bytes; + teardown-at-birth**: both pumps see instant EOF with zero bytes; the "spawn-and-forget" shape is the one bug class the R-02 telemetry (yield-once flag + birth-teardown debug hint) diagnoses at runtime. Hand-rolling the two-pump shape is out (ADR-050). -4. UDP: the plan payload is the FRAMED adapter (ADR-003) — the pump +4. UDP: the plan payload is the FRAMED adapter (ADR-003), so the pump is unchanged; the codec lives at the substrate/pump boundary. 5. **Bind-first listen handler (the accept dispatcher, ADR-008 §1):** the plan payload is the listener; after `accept_bi()`, the handler runs the accept loop (assembly-provided) concurrently with holding - the channel's `BiStream` — each accepted connection produces a + the channel's `BiStream`; each accepted connection produces a forwarded-channel open toward the listen opener. The listen channel's own stream stays idle (no pumps on it); its EOF cancels the loop (the listener tears down with the session). Per-accept - data planes are still `pump_bidi` (convention 8 — per-channel, + data planes are still `pump_bidi` (convention 8, per-channel, never a hand-rolled loop over one stream). **The handler is substrate-agnostic by construction:** it cannot know @@ -237,18 +237,18 @@ in-process pipe, and must not (AGENTS.md convention 7). Accept-as-open: when the bind-first listener accepts, the PRODUCER opens a channel back toward the listen opener. The opener SERVES the -op (`channels/tunnel/forwarded`, scope `tunnel:forwarded` — ingress; +op (`channels/tunnel/forwarded`, scope `tunnel:forwarded`, ingress; Amendment 2's two serving shapes: the recommended correlation-gate -establisher variant — a stale/missing token fails the open as a +establisher variant, where a stale/missing token fails the open as a typed `channel:open_failed`/`unknown_resource` before the channel -is born — or the base establisher-less shape, the data plane starts +is born, or the base establisher-less shape, where the data plane starts immediately and the assembly's post-open decline owns the miss): -1. The producer's per-accept open carries `{listen, substrate, peer}` - — the correlation token from the plan (the value the establisher +1. The producer's per-accept open carries `{listen, substrate, peer}`: + the correlation token from the plan (the value the establisher minted; the opener matches it against the `listen` field of its - open listener sessions — ADR-008 Amendment 1), the substrate - (also from the plan — stamped at establisher time from the one + open listener sessions per ADR-008 Amendment 1), the substrate + (also from the plan, stamped at establisher time from the one successful params parse; the dispatcher never re-parses wire input), the accepted connection's remote address (reply#2-class fidelity's carrier). @@ -256,19 +256,19 @@ immediately and the assembly's post-open decline owns the miss): adopts it (`TunnelSession::adopt`) and pumps the accepted halves against it (`pump_against`). 3. The opener-side handler correlates the open (via `listen`) to the - waiting session (assembly-side bookkeeping — the protocol provides + waiting session (assembly-side bookkeeping; the protocol provides the op and params, not the wait state machine), accepts the `BiStream`, and pumps against the local session's halves. Under the establisher variant (Amendment 2) that correlation lookup - rides the gate at establisher time instead — the handler-side + rides the gate at establisher time instead; the handler-side correlation remains only for the narrow residual race (a token vanishing between gate pass and handler run; the miss class is named `correlation-miss` at `debug`, assembly-side). 4. Identity: the forwarded op's opener-side gate sees the PRODUCER's - per-call identity (CF-005 precedence — the producer may carry an + per-call identity (CF-005 precedence; the producer may carry an `auth_token` when it is itself forwarding); the peer address in - params is producer-asserted data, not transport truth — the ACL - trusts the producer's grant, the application treats `peer` as + params is producer-asserted data, not transport truth; the ACL + trusts the producer's grant, and the application treats `peer` as metadata. ## Registration API (the spec surface) @@ -279,7 +279,7 @@ pub fn register_tunnel_openable( registry: &ResourceRegistry, // assembly-owned: (resource, substrate) -> backing on_registry: &Arc, // the session's dispatch registry // (the per-session fork per alkcall ADR-047 §4 - // amendment, 2026-09-03 — NOT the connection + // amendment, 2026-09-03, NOT the connection // overlay; Layer-2-registered open ops resolve // NOT_FOUND, review 004 F-01) auth: AuthContext, // the install-time context (per-call overlay is upstream's) @@ -310,7 +310,7 @@ pub fn register_tunnel_bindfirst_openable( // bind-first (shape 3, ADR-008) - `ResourceRegistry` (assembly construct, OQ-TN-11): maps `(resource, substrate)` → backing. The POCs' `HashMap` shape is the candidate; the collision domain is the producer's registry - (ADR-001 — two producers on one connection may expose the same + (per ADR-001, two producers on one connection may expose the same resource name independently). - The scope constants `TUNNEL_OPEN_SCOPE = "tunnel:open"`, `TUNNEL_DIRECT_SCOPE = "tunnel:direct"`, and @@ -321,7 +321,7 @@ pub fn register_tunnel_bindfirst_openable( // bind-first (shape 3, ADR-008) ## Access control posture -Tunnels reach local networks — the open gate is the security boundary +Tunnels reach local networks; the open gate is the security boundary (OQ-TN-08, ADR-006 + Amendment 1): - The registry runs `AccessControl` before the establisher (the ACL @@ -331,26 +331,26 @@ Tunnels reach local networks — the open gate is the security boundary `TUNNEL_FORWARDED_SCOPE` gates ingress (receiving forwarded connections) on the opener side. Three scopes, three capabilities. - The assembly layer MAY add resource-scoped checks via the - establisher (ownership on the resource name) — the protocol + establisher (ownership on the resource name); the protocol provides the seam, not the policy. For the direct op the seam is the parsed `TunnelTarget` (a deployment MAY enforce target - allowlists in its establisher — policy, not protocol). + allowlists in its establisher; policy, not protocol). - No target allowlists in the PROTOCOL, no tunnel-specific policy - machinery in v1 — the far side owns the resource (OQ-TN-08 + machinery in v1, as the far side owns the resource (OQ-TN-08 resolution); discovery is the ACL-filtered ops listing. - **Hub re-produce policy (the security model sentence, ADR-006 Amendment 1):** a hub re-producing the direct op applies ITS OWN - target policy at the re-produce hop — the hub's ACL governs the + target policy at the re-produce hop; the hub's ACL governs the final dial. The hub is the network boundary. ## Open Questions - **OQ-TN-11**: resource naming collision domain + lifecycle - (partially resolved — the per-producer registry is the leading + (partially resolved; the per-producer registry is the leading candidate) -- **OQ-TN-14**: resolved (split: unix in, stdio out — alktty's pipe - mode owns process stdio; see open-questions.md) -- **OQ-TN-15/16**: resolved (ADR-007/008 — the direct op, listen +- **OQ-TN-14**: resolved (split: unix in, stdio out, with alktty's pipe + mode owning process stdio; see open-questions.md) +- **OQ-TN-15/16**: resolved (ADR-007/008: the direct op, listen metadata, the forwarded op) ## References @@ -364,7 +364,7 @@ Tunnels reach local networks — the open gate is the security boundary - alkcall ADR-047/049 (open ops + establishment), ADR-050 (`pump_bidi`), ledger CF-005/CF-006 (identity precedence + per-call opener) - Upstream asks (review 008): `alkcall/docs/reviews/ - 008-tunnels-graduation-upstream-review.md` — both landed in + 008-tunnels-graduation-upstream-review.md`; both landed in alkcall 0.8.0 (flavor-form discovery derivation, ADR-047 amendment 3; the establisher → reply `bound` projection, ADR-049 amendment 3), plus the in-tree `ChannelRelay`/`HubLegTemplate` diff --git a/docs/architecture/wire.md b/docs/architecture/wire.md index 3155827..2eeacad 100644 --- a/docs/architecture/wire.md +++ b/docs/architecture/wire.md @@ -3,12 +3,12 @@ status: draft last_updated: 2026-09-20 (review 002 D-1/D-2 sync: the params schemas' port bounds, `additionalProperties: false` gates, `listen` `minimum: 1`, and the registry-side gate ordering recorded; ADR-007's oneOf wording corrected to the shipped allOf/if-then) --- -# alktunnels — Wire Format +# alktunnels: Wire Format The `alk/tunnel` wire surface. Two layers: the **open op** (JSON -params on the channels call plane — structured, schema-validated, +params on the channels call plane: structured, schema-validated, typed errors) and the **data plane** (bytes/datagrams inside the -channel `BiStream` — the codec, ADR-003). The channels layer strips +channel `BiStream`, the codec, ADR-003). The channels layer strips its own 8-byte header transparently (alknet ADR-093); the tunnel protocol owns everything inside the `BiStream`. @@ -21,7 +21,7 @@ this document is the normative WHAT. `Sub`-typed operation (alkcall ADR-047): the reply carries `{ "channel_id": }` once; the data plane flows on the channel's -`BiStream` — never in the call response stream. +`BiStream`, never in the call response stream. ### Input (params) @@ -32,33 +32,33 @@ this document is the normative WHAT. } ``` -- `resource`: string, required — the producer's stable resource name +- `resource`: string, required: the producer's stable resource name (NOT an address; the producer's registry maps it to its backing). -- `substrate`: string, required, enum `["tcp", "udp", "unix"]` — the +- `substrate`: string, required, enum `["tcp", "udp", "unix"]`: the extensible discriminator (ADR-001); selects the data-plane framing. The schema ships with all three values in v1 (unix has an - implementation path — OQ-TN-14). An older producer rejecting a - newer substrate value is the SSH "unknown channel type" posture — + implementation path, OQ-TN-14). An older producer rejecting a + newer substrate value is the SSH "unknown channel type" posture: loud, not silent. - `input_schema` pins both as required strings; unknown substrate - values fail schema validation (typed error — loud, not silent). + values fail schema validation (typed error, loud, not silent). - Extensibility: a new substrate is a new enum value, not a format change. An older producer rejecting a newer substrate is the SSH "unknown channel type" posture. ### Reply / errors -- Success: `{ "channel_id": }` — the channel is open, the +- Success: `{ "channel_id": }`: the channel is open, the establishment phase (dial) succeeded, the pump handler is running. Bind-first listen opens MAY add `"bound"` (§The Listen Open Op's Reply below). - Failure: a typed `channel:open_failed` call error with - `details: { reason, message }` — `reason ∈ dial_failed` / + `details: { reason, message }` with `reason ∈ dial_failed` / `unknown_resource` / `resource_shortage` / `handler_error` / `timeout` (alkcall ADR-049 §3). The registry's ACL gate runs before the wrapper (`FORBIDDEN` for a scope-less identity); the 256-channel cap is `channel:too_many_channels`. **A failed open never returns a - `channel_id`** — no phantom channel (the SSH contract, POC-verified + `channel_id`**; no phantom channel (the SSH contract, POC-verified from the consumer side). - The scope gate: the open op's `AccessControl.required_scopes` is `["tunnel:open"]` (the `TUNNEL_OPEN_SCOPE` constant). Caller @@ -69,10 +69,10 @@ this document is the normative WHAT. ## The Direct Op (`channels/tunnel/direct`) -Dynamic-target egress (ADR-007) — "dial what I name," not "dial what +Dynamic-target egress (ADR-007): "dial what I name," not "dial what the producer registered." `Sub`-typed, ALPN marker `alk/tunnel`; the same establishment phase and wrapper as `…/sub`. Scope gate: -`["tunnel:direct"]` (`TUNNEL_DIRECT_SCOPE`) — **never implied by +`["tunnel:direct"]` (`TUNNEL_DIRECT_SCOPE`), **never implied by `tunnel:open`** (ADR-006 Amendment 1). ### Input (params) @@ -84,42 +84,42 @@ same establishment phase and wrapper as `…/sub`. Scope gate: } ``` -- `substrate`: string, required, enum `["tcp", "udp", "unix"]` — the +- `substrate`: string, required, enum `["tcp", "udp", "unix"]`: the same discriminator semantics as `…/sub` (ADR-001); selects the data-plane framing. -- `target`: object, required, keyed by the sibling `substrate` — +- `target`: object, required, keyed by the sibling `substrate`: - `tcp`/`udp` → `{ "host": , "port": }` (domain-form - hosts stay unresolved on the wire — producer-side resolution, the + hosts stay unresolved on the wire; producer-side resolution, the ssh `-D` semantic); - `unix` → `{ "path": }`. - `input_schema` pins both required; the target sub-schema is - per-substrate — an `allOf` of two `if`/`then` branches keyed on the + per-substrate: an `allOf` of two `if`/`then` branches keyed on the sibling `substrate` value, validated **registry-side** (the schema - gate runs before the establisher — the behavior + gate runs before the establisher, as the behavior `malformed_target_is_schema_class_and_leaves_no_channel` pins; the establisher's serde parse re-checks as the backstop). Malformed targets are the `invalid_input`-class failure; **`unknown_resource` can never fire on the direct op** (no registry lookup). -- Schema bounds (wire-stable — the target sub-schema is part of the +- Schema bounds (wire-stable; the target sub-schema is part of the one-way door, ADR-007 §"Wire-stable"): each branch's target object - carries `"type": "object"` and `"additionalProperties": false` — + carries `"type": "object"` and `"additionalProperties": false`: - `tcp`/`udp` → `{ "host": {"type": "string"}, "port": {"type": "integer", "minimum": 0, "maximum": 65535} }`, both required; - `unix` → `{ "path": {"type": "string"} }`, required. The `additionalProperties: false` gate is what makes a cross-shape target (a `unix` open carrying `host`) fail the registry's - `INVALID_INPUT` class *before* the establisher — not a serde-level + `INVALID_INPUT` class *before* the establisher, not a serde-level quirk. ### Reply / errors -- Success: `{ "channel_id": }` — identical semantics to +- Success: `{ "channel_id": }`: identical semantics to `…/sub`; no `bound` field (dial establishers do not bind). - Failure: the same typed surface as `…/sub` minus `unknown_resource`, plus the schema/`invalid_input` class for malformed targets. -## The Listen Open Op's Reply — the `bound` and `listen` Fields +## The Listen Open Op's Reply: the `bound` and `listen` Fields A bind-first listen open (ADR-008 §1, served by `…/sub` with the bind-first establisher) MAY add optional, additive reply fields: @@ -133,16 +133,16 @@ bind-first establisher) MAY add optional, additive reply fields: (`{host, port}` / `{path}`); the REAL OS-chosen address (port `0` resolved). - `listen` is the correlation token (ADR-008 Amendment 1): a u32 the - establisher minted, unique per open on that producer — the key the + establisher minted, unique per open on that producer; the key the forwarded op's params carry (§The Forwarded Op below). Riding the reply is what makes it relay-transparent: reply fields pass through hub re-produce untouched (alkcall ADR-051), and the same minted token reappears in every hop's reply. - Present only when the establisher supplies them; absent for dial establishers and accept-first listen opens. Old consumers ignore - unknown reply fields (plain JSON) — wire-additive. + unknown reply fields (plain JSON), which is wire-additive. - `bound` is informational surface, not addressing: a consumer MUST - NOT treat `bound` as a dial target (the producer owns the backing — + NOT treat `bound` as a dial target (the producer owns the backing, ADR-001). It exists for RFC-1928-reply-class fidelity (BND.ADDR) and observability. @@ -150,17 +150,17 @@ bind-first establisher) MAY add optional, additive reply fields: Accept-as-open (ADR-008 §3): when a bind-first listener accepts, the PRODUCER opens a channel back toward the listen-opener side. The -producer *calls* the op (the opener SERVES it — the ADR-047 §5 +producer *calls* the op (the opener SERVES it, per the ADR-047 §5 connection-owner rule: the serving side allocates the `channel_id`, the producer-side initiator adopts). `Sub`-typed, ALPN marker `alk/tunnel`; the serving side registers establisher-less by base -shape (the data plane starts immediately) or — the recommended shape, -ADR-008 Amendment 2 — with the correlation-gate establisher variant +shape (the data plane starts immediately) or (the recommended shape, +ADR-008 Amendment 2) with the correlation-gate establisher variant (a stale/missing token then fails the op as a typed `channel:open_failed` with `details.reason = "unknown_resource"`; the wire vocabulary is unchanged by the amendment). Scope gate (opener side): `["tunnel:forwarded"]` -(`TUNNEL_FORWARDED_SCOPE`) — ingress, a separate grant from +(`TUNNEL_FORWARDED_SCOPE`): ingress, a separate grant from `tunnel:open`/`tunnel:direct` (ADR-006 Amendment 1). ### Input (params) @@ -173,23 +173,23 @@ Scope gate (opener side): `["tunnel:forwarded"]` } ``` -- `listen`: integer > 0, required — the correlation token from the +- `listen`: integer > 0, required: the correlation token from the listen open's `listen` reply field (ADR-008 Amendment 1; the producer-minted token, relay-transparent). The opener correlates the accept to its waiting listener session by matching it. The schema pins `"minimum": 1` and the serde type pins it as the backstop (a `listen: 0` on the wire is a malformed open, rejected - at parse time — never silently accepted). -- `substrate`: string, required, enum — the listen tunnel's + at parse time, never silently accepted). +- `substrate`: string, required, enum: the listen tunnel's substrate; shapes the forwarded channel's data-plane framing. -- `peer`: object, required — the accepted connection's remote address, +- `peer`: object, required: the accepted connection's remote address, substrate-shaped (`{host, port}` / `{path}`). Reply#2-class fidelity's carrier (ask #2). - Schema bounds (wire-stable, mirroring the direct op's target sub-schema): the per-substrate `peer` sub-schema is an `allOf` of two `if`/`then` branches keyed on the sibling `substrate`, each - branch's object carrying `"type": "object"` and - `"additionalProperties": false` — + branch's object carrying `"type": "object"` and + `"additionalProperties": false`: - `tcp`/`udp` → `{ "host": {"type": "string"}, "port": {"type": "integer", "minimum": 0, "maximum": 65535} }`, both required; - `unix` → `{ "path": {"type": "string"} }`, required. @@ -199,22 +199,22 @@ Scope gate (opener side): `["tunnel:forwarded"]` ### Reply / errors -- Success: `{ "channel_id": }` — the opener-allocated ID the +- Success: `{ "channel_id": }`: the opener-allocated ID the producer adopts, then pumps the accepted halves against. - Failure: a stale/missing correlation token is the typed `channel:open_failed` with `details.reason = "unknown_resource"` - (the establisher variant, ADR-008 Amendment 2 — the recommended - serving shape; the miss never births a channel). An opener without + (the establisher variant, ADR-008 Amendment 2, which is the + recommended serving shape; the miss never births a channel). An opener without the scope is `FORBIDDEN`; an old opener that does not serve the op - rejects loudly (`NOT_FOUND` — the SSH posture). Other open failures + rejects loudly (`NOT_FOUND`: the SSH posture). Other open failures ride the existing typed vocabulary. Under the base (establisher- - less) serving shape an in-session miss produces no call error — it + less) serving shape an in-session miss produces no call error; it is the assembly-side post-open decline (the channel reaps at birth, `correlation-miss` at `debug`). ## The Data Plane -One data stream per direction on one channel — the channel ID is the +One data stream per direction on one channel: the channel ID is the flow key (no conn-id, no sub-demux). Two codecs by substrate (ADR-003): @@ -225,7 +225,7 @@ overhead is the channels 8-byte chunk header. A zero-length read is genuinely EOF (stream semantics). Both pumps run `alkcall::channels::pump_bidi(channel, peer_read, -peer_write)` — the two-pump contract (alknet ADR-078, pinned upstream +peer_write)`, the two-pump contract (alknet ADR-078, pinned upstream ADR-050): when one direction's source EOFs, the pump shuts the OPPOSITE sink down (the EOF sentinel crosses the mux; the peer sees a clean half-close), and the helper completes when both pumps finish, @@ -234,15 +234,15 @@ returning `(u64, u64)` copy counts. Copy errors are EOF-shaped ### Datagram substrate (`udp`): mandatory length framing -Every datagram rides `[len: u16 BE][datagram]` on BOTH directions — -mandatory for correctness (the F-2 rationale lives in ADR-003; the +Every datagram rides `[len: u16 BE][datagram]` on BOTH directions; +this is mandatory for correctness (the F-2 rationale lives in ADR-003; the short form: raw pass-through cannot carry an empty datagram). Under the codec: - **Framing (`frame_datagram`):** `len: u16 BE` + payload. Datagrams - > 65535 bytes are rejected at frame time (`Oversize`) — never a + > 65535 bytes are rejected at frame time (`Oversize`), never a wire overflow (the u16 length field would wrap). -- **Decoding (`DatagramReader`):** incremental — datagrams split +- **Decoding (`DatagramReader`):** incremental; datagrams split across channel chunks, batch into single chunks, and survive partial headers. The decoder buffers state across reads; it is the only consumer-visible decode path. @@ -253,7 +253,7 @@ the codec: returns `Some(bytes)` per datagram (possibly empty), `None` only on stream EOF. - **Chunk-size independence:** datagram boundaries survive chunk - splitting, batching, and any channels-layer re-chunking — verified + splitting, batching, and any channels-layer re-chunking, verified end-to-end at 7-byte chunk splits (forward POC). - **MTU discipline:** 1400-byte datagrams (a conservative MTU-safe payload, not the theoretical 1472 max) ride the bounded-buffer path; @@ -262,10 +262,10 @@ the codec: - **Truncation:** silent truncation never happens (OQ-TN-13, resolved fail-loud per ADR-003): the `local` adapter recvs on a full 65535-byte scratch (UDP `try_recv` truncates silently at the buffer - size — a smaller scratch would corrupt the framing) and a + size; a smaller scratch would corrupt the framing) and a >caller-buffer frame is staged across reads, the length prefix intact, so the far-side de-framer reassembles the true length. A - wire-level over-bound datagram (>65535 — impossible for a + wire-level over-bound datagram (>65535, impossible for a well-formed peer) is a loud error at the frame path. ### Sentinels and lifecycle (both codecs) @@ -282,10 +282,10 @@ the codec: idle-expiry mapping is a producer-registry concern, not wire). - **No establishment frame.** The open op's reply IS the establishment result (ADR-049); the pre-ADR-049 - "JSON ack before data" hunch is superseded — the wire carries zero + "JSON ack before data" hunch is superseded; the wire carries zero tunnel-level control framing in v1. - **BAST:** the binary framing (the UDP codec) carries its BAST - document — [bast.md](bast.md) (AGENTS.md convention 12; the + document in [bast.md](bast.md) (AGENTS.md convention 12; the stream pass-through has no binary framing to describe). ## Normative byte diagrams @@ -307,11 +307,11 @@ UDP substrate, one direction's wire view: ## Open Questions -- **OQ-TN-13**: resolved (fail-loud, ADR-003) — see the Truncation +- **OQ-TN-13**: resolved (fail-loud, ADR-003); see the Truncation bullet -- **OQ-TN-02**: resolved — see the ADR-003 mandate -- **OQ-TN-15**: resolved — see ADR-007 (the direct op) -- **OQ-TN-16**: resolved — see ADR-008 (the `bound`/`listen` fields + +- **OQ-TN-02**: resolved; see the ADR-003 mandate +- **OQ-TN-15**: resolved; see ADR-007 (the direct op) +- **OQ-TN-16**: resolved; see ADR-008 (the `bound`/`listen` fields + the forwarded op) ## References @@ -333,5 +333,5 @@ UDP substrate, one direction's wire view: ADR-050 (`pump_bidi`); alknet ADR-071/093 (channels wire) - `docs/research/tunnels-graduation.md` (the graduation research the new ops implement); the upstream asks (review 008) landed in - alkcall 0.8.0 — see + alkcall 0.8.0; see `alkcall/docs/reviews/008-tunnels-graduation-upstream-review.md` \ No newline at end of file