diff --git a/docs/architecture/README.md b/docs/architecture/README.md new file mode 100644 index 0000000..daba6c5 --- /dev/null +++ b/docs/architecture/README.md @@ -0,0 +1,147 @@ +--- +status: draft +last_updated: 2026-09-07 +--- + +# alktunnels — Architecture + +Arbitrary bidirectional tunnels over alkcall channels: the +`alk/tunnel`-family producer/consumer protocol crate. This directory +holds the architecture spec docs and the ADRs. It is the sibling of +alktty's architecture (`/workspace/@alkdev/alktty/docs/architecture/`) +— same substrate (alkcall channels), same shape (a producer +open-handler + a typed consumer session), different service (tunnels: +raw byte/datagram transport, not terminal sessions). + +## Documents + +| Document | Status | Description | +|----------|--------|-------------| +| [overview.md](overview.md) | draft | Crate purpose, the resource model in brief, dependencies, ALPN, feature gates, module map | +| [wire.md](wire.md) | draft | The wire format: open-op params JSON (ADR-001), the data-plane codec (raw pass-through for stream substrates; mandatory `[len: u16 BE]` framing for UDP — ADR-003), sentinels, half-close semantics | +| [producer.md](producer.md) | draft | The producer half: the `channels/tunnel/sub` open op, the establisher (dial + listen variants), the pump handler, registration, scope gating | +| [consumer.md](consumer.md) | draft | The consumer half: `TunnelSession` (stream + datagram variants), the reverse-flow initiator, teardown ownership (ADR-005) | +| [bast.md](bast.md) | draft | The BAST (Binary Abstract Syntax Tree) document for the data-plane's binary framing (the UDP `[len: u16 BE]` codec — ADR-003); the stream pass-through has no binary framing to describe | +| [open-questions.md](open-questions.md) | draft | Centralized OQ tracker (OQ-TN-01..14; the Phase 0 ledger promoted) | + +## Applicable ADRs + +alktunnels-native ADRs (001..005). Upstream ADRs (alkcall/alknet) are +referenced by their own numbers and are NOT ported — the alkcall +originals at `/workspace/@alkdev/alkcall/docs/architecture/decisions/` +remain authoritative for the shared surfaces (establishment, channels +wire format, `pump_bidi`, identity seam). + +| ADR | Title | Status | +|-----|-------|--------| +| [001](decisions/001-open-params-layout.md) | Open-Op Params Layout: Resource + Substrate | Accepted | +| [002](decisions/002-alpn-strategy.md) | Single `alk/tunnel` ALPN (Option A) | Accepted | +| [003](decisions/003-codec-and-udp-framing.md) | Data-Plane Codec: Raw Pass-Through (Stream) / Mandatory Length Framing (UDP) | Accepted | +| [004](decisions/004-no-backend-trait.md) | No TunnelBackend Trait — Halves Functions at the Assembly Layer | Accepted | +| [005](decisions/005-consumer-session-owns-teardown.md) | The Consumer Session Type Owns Teardown | Accepted | +| [006](decisions/006-access-control-posture.md) | Access-Control Posture — the Open Gate Is the Security Boundary | Accepted | + +## Key Design Principles + +1. **A tunnel is a resource, not an address.** `params` identify a + produced resource + substrate discriminator; the producer owns the + backing (local port, docker container, in-process service, unix + socket). No URL-style general addressing in the wire — rich + addressing enters only through `-D`/dynamic composition, which + speaks its protocol inside the tunnel payload. See + [ADR-001](decisions/001-open-params-layout.md) and + [wire.md](wire.md). + +2. **Role follows the resource; connection direction is independent.** + Whoever can reach the target is the producer (registers openable + channels); whoever wants the bytes is the consumer. `-L` and `-R` + are the same open op with the entry point on different machines — + no protocol-level direction (the hub-owns-the-connection model, + OQ-TN-03). Validated by the reverse-flow POC: the worker serves on + the connect side; the hub initiates per accept. See + [producer.md](producer.md) and [consumer.md](consumer.md). + +3. **Raw pass-through for stream substrates; mandatory length framing + for UDP.** A tunnel has one data stream per direction — no + sub-demux key, no tunnel-level header on stream substrates (0 B + tunnel overhead; the channels 8-byte header is the only cost). + UDP rides `[len: u16 BE]` framing — mandatory for correctness + (F-2; the rationale and the executable pin live in ADR-003). See + [ADR-003](decisions/003-codec-and-udp-framing.md) and + [wire.md](wire.md). + +4. **The two-pump shape is `alkcall::channels::pump_bidi` — never + hand-rolled.** The producer's pump handler and the consumer-side + pump both consume the upstream helper (ADR-050); each pump shuts + down the opposite sink on completion (half-close semantics fall + out). The `OpenHandler`'s returned `JoinHandle` tracks the + data-plane lifetime (early return = teardown-at-birth, R-02). See + [producer.md](producer.md). + +5. **No backend trait.** Stream substrates need "produce boxed halves + for a resource" — a function, not a trait (the UDP POC's OQ-TN-05 + finding). The inversion point is the assembly layer wiring + substrate halves into the establisher/pump; substrate types stay + in feature-gated backend modules. The listen variant (producer + accepts instead of dialing) is an establisher shape, not a trait. + See [ADR-004](decisions/004-no-backend-trait.md). + +6. **The consumer session type owns teardown.** One session = one + channel = one tunnel (the channel ID is the flow key). The adopted + side has no wrapper awaiting its pump (reverse POC W3) — the + session must hold the pump handle and reap the channel on + completion/close. See [ADR-005](decisions/005-consumer-session-owns-teardown.md) + and [consumer.md](consumer.md). + +7. **Backpressure and limits are inherited, not redefined.** Bounded + per-channel buffers, the 256-channel cap, monotonic IDs, and the + zero-length-sentinel EOF convention are alkcall channels + invariants; the tunnel crate consumes them and does not build a + second demux/mux (AGENTS.md convention 10). + +8. **Security posture: the open gate is the boundary.** Scope-gated + opens (`TUNNEL_OPEN_SCOPE = "tunnel:open"`, stable once published + — ADR-006), alkcall's op-level ACL via `register_openable`; caller + identity resolves in the 0.7.0 precedence order (token > + `ServingConfig.identity` > transport); the establisher/pump + handler see the per-call opener identity. No target allowlists, no + tunnel-specific policy machinery in v1. See + [ADR-006](decisions/006-access-control-posture.md) and + [producer.md](producer.md). + +9. **Wasm-clean default crate.** The default crate (protocol only) + compiles to `wasm32-unknown-unknown`; socket/platform I/O lives in + feature-gated backend modules (AGENTS.md conventions 4/14), same + as alktty. + +## Relevant Open Questions + +All open questions are tracked in +[open-questions.md](open-questions.md). The Phase 0 OQ ledger +(OQ-TN-01..10, `docs/research/phase-0-findings.md`) is promoted there; +the questions affecting the spec set are summarized in each doc's Open +Questions section. + +## References + +- Phase 0 research: `docs/research/phase-0-findings.md` (OQ ledger), + `docs/research/poc-summary.md` (UDP POC), + `docs/research/reverse-poc-summary.md` (reverse-flow POC), + `docs/research/ssh-socks5-survey.md` (SSH/SOCKS5/udpgw prior art) +- Upstream: alkcall ADR-047 (openable ALPNs are operations), ADR-049 + (establishment phase + amendment 2 plan payload), ADR-050 + (`pump_bidi`), ADR-022 §2 (both-sides serving), ADR-047 §5 + (connection-owner allocation), ledger CF-005/CF-006 (identity seam); + alknet ADR-078 (two-pump contract), ADR-071/093 (channels wire + format); alkcall ADR-042 (hub relay) +- Upstream reviews/ledger (the spec cites these IDs): + `/workspace/@alkdev/alkcall/docs/reviews/007-establishment-follow-ups-review.md` + (R-01 = the `Establishment` plan payload, R-02 = the `OpenHandler` + lifetime contract, R-03 = `pump_bidi`) and + `/workspace/@alkdev/alkcall/docs/reviews/consumer-findings-ledger.md` + (CF-005/CF-006 = the identity seam, resolved in 0.7.0) +- Sibling precedent: alktty (`/workspace/@alkdev/alktty`) — the + producer/consumer-on-channels shape, wasm-clean default, + feature-gated local backend, BAST convention +- POC code: `/workspace/alktunnels-udp-poc` (forward), + `/workspace/alktunnels-reverse-poc` (reverse) \ No newline at end of file diff --git a/docs/architecture/bast.md b/docs/architecture/bast.md new file mode 100644 index 0000000..3d89d5e --- /dev/null +++ b/docs/architecture/bast.md @@ -0,0 +1,109 @@ +--- +status: draft +last_updated: 2026-09-07 +--- + +# alktunnels — BAST Document for the `alk/tunnel` Wire Format + +This document is the **BAST (Binary Abstract Syntax Tree)** +specification for the binary portions of the `alk/tunnel` wire format +— per AGENTS.md convention 12, the mandatory UDP length-framing codec +([ADR-003](decisions/003-codec-and-udp-framing.md)) is binary framing +beyond pass-through, so the BAST document exists even though the +format is trivial. It is a normative JSON document conforming to the +BAST meta-schema at `https://alk.dev/bast/v1/schema` (JSON Schema +Draft 2020-12); any Draft 2020-12 validator can check well-formedness. + +**alktunnels does not depend on alktype.** The hand-rolled +`frame_datagram` / `DatagramReader` (the `src/wire.rs` shape the POCs +ran) is the runtime codec; this BAST document is the human-readable +contract describing what those functions round-trip. If runtime +validation against the BAST becomes desirable later, alktype becomes +an optional dep and this document is there to feed it. + +## Scope + +The `alk/tunnel` wire format is: a JSON open op on the channels call +plane (a UTF-8 JSON object inside the channels envelope — not a +binary layout; specified in [wire.md](wire.md) §The Open Op and +implemented by `TunnelParams`/`tunnel_open_spec` in the Rust source), +and a binary data plane inside the channel `BiStream`. + +Per the BAST format spec, a BAST document describes **binary data +layouts** — not JSON payloads +([`validate_bytes` vs `validate_json`](https://alk.dev/bast/v1/spec)). +This document therefore covers only the data plane's binary framing: + +1. **The datagram frame** (`[len: u16 BE][datagram bytes]`) — the + UDP-substrate codec (ADR-003; mandatory, F-2). Stream substrates + ride raw pass-through — zero tunnel-level framing, so there is + nothing for BAST to describe there. + +What this BAST deliberately does **not** cover: + +- **The open-op params JSON shape** (`{resource, substrate}`) — UTF-8 + JSON on the call plane; specified in + [ADR-001](decisions/001-open-params-layout.md) and + [wire.md](wire.md), implemented by `TunnelParams`. +- **The channels 8-byte chunk header** — the channels layer's own + framing (alknet ADR-071/093); stripped transparently before the + tunnel data plane sees bytes. Specified upstream, not here. +- **The EOF sentinel** — the channels-level zero-length chunk + (upstream); the tunnel codec never emits it (that is the F-2 + invariant). + +## The BAST document + +```json +{ + "$schema": "https://alk.dev/bast/v1/schema", + "$defs": { + "DatagramFrame": { + "kind": "struct", + "endian": "big", + "fields": [ + { "name": "len", "kind": "uint16" }, + { "name": "datagram", "kind": "bytes", "encoding": "length-prefixed", "maxLength": 65535 } + ] + } + } +} +``` + +## Annotations + +### `DatagramFrame` — binary, exact + +Maps 1:1 to the on-wire bytes. `len` as `uint16` big-endian (2 bytes) +followed by exactly `len` bytes of datagram payload. The zero-byte +payload (`len == 0`) is a legal empty datagram — a real datagram, NOT +EOF (the F-2 invariant; EOF is the channels-level sentinel, outside +this codec). The `maxLength: 65535` encodes the u16 bound; the codec +rejects framing a larger datagram at frame time (`Oversize`) rather +than wrapping the length field. + +One frame per datagram, in both directions of the channel +`BiStream`. Frames are self-delimiting on the byte stream; a decoder +is an incremental state machine across chunk boundaries (chunk +splitting/batching is transparent — forward POC, 7-byte chunk splits +verified). + +### Layering note + +The codec sits at the substrate/pump boundary (the establisher wraps +the dialed UDP half in the framed adapter — ADR-003's placement, so +the pump handler stays substrate-agnostic). The BAST describes the +wire inside the `BiStream` only; the pump/pump_bidi contract +(ADR-050) is upstream's, not re-specified here. + +## References + +- [ADR-003](decisions/003-codec-and-udp-framing.md) (the codec + decision + F-2 mandate), [ADR-001](decisions/001-open-params-layout.md) + (the JSON open-op shape — outside BAST scope) +- [wire.md](wire.md) (the normative prose; §Normative byte diagrams) +- alktty `tty-bast.md` (the BAST precedent — same meta-schema, same + "binary framing only" scope rule) +- Forward POC `docs/research/poc-summary.md` §3 (the codec's + validation matrix); reverse POC + `docs/research/reverse-poc-summary.md` §F-2 (the mandate) \ No newline at end of file diff --git a/docs/architecture/consumer.md b/docs/architecture/consumer.md new file mode 100644 index 0000000..45dce91 --- /dev/null +++ b/docs/architecture/consumer.md @@ -0,0 +1,154 @@ +--- +status: draft +last_updated: 2026-09-07 +--- + +# alktunnels — Consumer Half + +The consumer half: `TunnelSession` — the typed client for tunnel +channels. The consumer is whoever wants the bytes; it opens tunnel +channels toward the producer (forward path), or per local accept +toward a connect-side serving producer (the reverse flow). Both are +the same session type — role follows the resource, not the connection +(OQ-TN-03). Design decisions: [ADR-005](decisions/005-consumer-session-owns-teardown.md) +(session owns teardown), [ADR-003](decisions/003-codec-and-udp-framing.md) +(codec); this document is the normative WHAT. + +## Opening a Tunnel + +### Forward path (`-L`/direct — the consumer dials the producer's transport) + +```rust +let session = TunnelSession::open(&channel_client, params).await?; +``` + +- `params`: `{resource, substrate}` (ADR-001) — the produced resource + + substrate discriminator. +- Flow: `ChannelClient::open_channel("channels/tunnel/sub", params, + "alk/tunnel")` → adopt the returned channel ID → split the channel + `BiStream` → present the substrate-shaped data plane. +- Errors: a failed open resolves a typed error — + `ChannelOpenError::CallFailed` carrying the wire `CallError`; + branch on `establishment_reason()` for `channel:open_failed`'s + reason (dial_failed / unknown_resource / resource_shortage / + handler_error / timeout), or the pre-establishment codes + (`FORBIDDEN`, `channel:too_many_channels`). **A failed open never + yields a session** — no phantom session, mirroring the + no-phantom-channel property (POC-verified). + +### Reverse path (`-R` — the consumer initiates toward a connect-side serving producer) + +The hub/accept-side shape (the reverse POC's `ReverseTunnel`, the +seed of this path). Construction differs from the forward path only +in how the channel is obtained — the session type and teardown API +are identical: + +```rust +// Per local accept (assembly-owned listener): +let channel_id = open_reverse_channel(&hub_call, ¶ms, auth_token).await?; +let session = TunnelSession::adopt(hub_manager, channel_id, TUNNEL_ALPN).await?; +let pump_handle = session.pump_against(accepted_halves).await; // or drive the halves directly +``` + +- `open_reverse_channel` — call the worker's open op on channel 0 + (`call_with_payload`; optional `auth_token`), extract the + worker-allocated `channel_id`. Caller identity rides the transport + by default (alkcall 0.7.0 CF-005 (b): `Connection::set_identity` + before dialing); the payload `auth_token` is the optional + hub-forwarding path (precedence: token > `ServingConfig.identity` > + transport). +- `TunnelSession::adopt(manager, channel_id, alpn)` — adopt the + worker-allocated ID (connection-owner rule, ADR-047 §5 — the + SERVING side allocates; the initiator adopts; early-arrival parking + covers the adoption race), install the session's data plane from + the adopted halves. `adopt` takes the same typed-error surface as + `open` (an adopt failure is `AdoptFailed`-class, not an + establishment error). +- `pump_against(accepted_halves)` — spawn `pump_bidi` against the + accepted local connection and hold the pump handle (ADR-005). + +The session type and its teardown API are identical on both paths — +one API, `-L` and `-R` alike. + +## The Data Plane (substrate-shaped) + +- **Stream variant (`tcp`, `unix`):** `stream_halves()` — raw + `AsyncRead`/`AsyncWrite` halves; the halves ARE the tunnel (raw + pass-through, ADR-003). For owned access (spawning local pumps), + `take_halves()` consumes the session's halves. +- **Datagram variant (`udp`):** `send_datagram(&[u8])` / + `recv_datagram() -> Option` over the mandatory + `[len: u16 BE]` codec (ADR-003). `recv_datagram` returns + `Some(bytes)` per datagram — possibly empty (`len=0` is a legal + empty datagram) — and `None` only on stream EOF (the channels-level + sentinel; the codec never collides with it, F-2). A >65535-byte + send is rejected at frame time (`Oversize`). + +## Lifecycle and Teardown (the point of ADR-005) + +One session = one channel = one tunnel. The session owns: + +- **The adopted channel entry** — nothing upstream awaits the + adopter's pump (the W3 gap); the session closes it structurally. +- **The session-owned pump handle** (when the session spawned one — + the reverse path's `pump_against`). + +Teardown API: + +- `close(self)` — abort the pump (if session-owned), tear down the + adopted channel (`ChannelManager::teardown_channel`), consume the + session. The ungraceful path. +- `join(self)` — await pump completion (both directions finished; + half-close semantics fall out of `pump_bidi` — one direction EOFs, + the opposite sink shuts down, the other keeps pumping until its own + EOF; W4-validated), THEN reap the adopted channel. Returns the + `(u64, u64)` copy counts for observability. The graceful path. + **Pump-less sessions** (after `take_halves`, or a datagram session + the caller drives directly): there is no session-owned pump — + `join` completes immediately (nothing to await) and reaps only; + the copy counts are `(0, 0)`. +- `Drop` — aborts the pump and sync-reaps the channel (teardown + semantics of `close`, best-effort — `Drop` cannot await). Dropping + without close/join never leaks the channel entry. +- **`TunnelSession` does not implement `Clone`.** Aliasing a session + would alias its teardown; multi-channel consumers hold a + `Vec` (or the assembly layer does). + +## Out-of-band close + +A peer-initiated `channel/close` (the generic channel op on the +producer's serving registry) tears the SERVING side down; the +consumer's session still owns its adopted entry — `close`/`join` reaps +it. The reverse POC pinned this split (the close ran on the worker; +the hub reaped its own entry, W3). EOF propagation is symmetric: +either side's teardown reaches the other as the channels-level +sentinel. + +## Both-Sides Sanity + +The consumer and producer halves compose on one connection in both +roles (alkcall ADR-022 §2): a worker dials the hub (connect side), +serves its open ops (serving side), and its own outbound calls still +resolve while serving (reverse POC: worker_outbound_calls_still_ +resolve_while_serving). The consumer half adds nothing to the +connection's serving posture — a pure consumer (`from_connection`) +never dispatches inbound requests; a reverse-flow initiator needs no +serving registry of its own. + +## Open Questions + +- **OQ-TN-12**: hub re-produce composition (deferred — a hub proxy + re-exporting a consumed resource would add a + `register_openable_with_establisher` whose establisher opens a + consumer channel; no concrete consumer yet) +- **OQ-TN-13**: UDP truncation (resolved — fail-loud, ADR-003) + +## References + +- [producer.md](producer.md) (the serving side's shapes), + [wire.md](wire.md) (the data plane) +- Reverse POC `docs/research/reverse-poc-summary.md` (the seed shape: + `ReverseTunnel` open_and_pump / close / join_and_reap; W3, W4) +- alkcall ADR-047 §5 (allocation — the initiator adopts), ADR-050 + (`pump_bidi`), ledger + CF-005/CF-006 (identity on the reverse path) \ No newline at end of file diff --git a/docs/architecture/decisions/001-open-params-layout.md b/docs/architecture/decisions/001-open-params-layout.md new file mode 100644 index 0000000..28717bf --- /dev/null +++ b/docs/architecture/decisions/001-open-params-layout.md @@ -0,0 +1,120 @@ +# ADR-001: Open-Op Params Layout — Resource + Substrate + +## Status + +Accepted (2026-09-07) + +## Context + +The tunnel open op's `params` (the `input` of the +`channels/tunnel/sub` operation, alkcall ADR-047) is the wire-stable +carrier of "what to open." alknet ADR-071 §ALPN table noted +`alknet/tunnel` as `[0, 1]` data in/out only; the addressing scheme +was never decided there. Phase 0 resolved the framing of the problem +(the OQ-TN-01 reframe, 2026-09-05): the rich "remote addressing" +framing was an XY problem — except in the `-D`/dynamic case (which +composes at the assembly layer), a tunnel is either TCP or UDP, and +`params` need only *identify a produced resource*, not carry a +general-purpose address. + +Constraints that shaped the decision: + +- `params` is ALPN-specific JSON, interpreted by the open op's + establisher/handler — never by the channels layer (alkcall + ADR-039). The layout is a one-way door once a consumer exists + (wire-stable once published; additive changes only). +- The identifier must be substrate-extensible without format changes + (a new substrate is an additive `substrate` value, not a v2 + format). SSH's `direct-streamlocal` (same open-op shape, degenerate + address slots, new type string) and SOCKS5's ATYP (scheme-tagged + addresses) are the prior-art anchors; both support the "one + identifier + one discriminator" reduction. +- The producer owns the backing: the consumer never learns an + address. A `host:port` in params would leak the producer's + topology into the wire and pin the wire to address-shaped targets. +- alkcall's registry schema-validates the open op's input + (`input_schema` runs before the establisher — review 007 Part B), + so the layout must be schema-describable (no per-substrate + polymorphic payloads in v1). +- Both POCs used exactly this shape (`TunnelParams {resource, + substrate}`); 17 + 16 tests rode it end-to-end. + +## Decision + +`params` for the `channels/tunnel/sub` open op is a self-contained +JSON object: + +```json +{ + "resource": "postgres-primary", + "substrate": "tcp" +} +``` + +- **`resource`** (string, required) — the produced resource + identifier: the producer's stable name for the tunnel target. NOT + an address; the producer's registry (an assembly-side construct, + OQ-TN-11) maps the name to its backing (a local port, a docker + container's port, an in-process service, a unix socket path). +- **`substrate`** (string, required) — the extensible discriminator: + `"tcp"` | `"udp"` | `"unix"` in v1. A new substrate is a new value, + not a format change — the same additive property SSH gets from + channel-type strings and SOCKS5 gets from ATYP values. The + discriminator also selects the data-plane framing (ADR-003): the + establisher and both pumps must agree on it, so it rides the open + op rather than being inferred. +- The open op's `input_schema` pins both fields as required strings; + `substrate` is enum-validated against the v1 set. Unknown + substrate values are a schema failure (typed open error) — a + consumer speaking a newer substrate to an older producer fails + loudly, not silently. +- Substrate-specific detail (the path of a unix socket, the UDP + associate semantics) is owned by the producer's registry, not the + wire. The resource name is opaque to the protocol. +- The open-failure error path rides alkcall ADR-049 (typed + `channel:open_failed` with `details.reason` ∈ `dial_failed` / + `unknown_resource` / `resource_shortage` / `handler_error` / + `timeout`): an unknown resource name is `unknown_resource` — no + params-level error field, no establishment frame (the pre-ADR-049 + OQ-TN-09 hunch is superseded). + +Rich in-band addressing (SOCKS5 ATYP, per-datagram remote addresses) +enters only through the `-D`/dynamic-target composition path — it is +carried INSIDE the tunnel payload by whatever protocol the tunnel +carries (socks5, udpgw-style framing), never in base params. + +## Consequences + +- **Wire-stable from the first consumer.** Both fields are required; + renames are a breaking wire change (do not). Adding an optional + field is additive-safe; adding a `substrate` value is + consumer-progressive (an older producer rejects with + `invalid_input`-class schema error — the SSH "unknown channel type" + posture, loud not silent). +- **The BAST question is moot for params.** Params is JSON inside the + channels open op (which is already a JSON envelope); the data + plane's binary framing (ADR-003) carries its BAST doc at + `docs/architecture/bast.md`. Per AGENTS.md convention 12, the + hand-rolled codec remains the runtime implementation with the BAST + doc as the contract. +- **Producers own a naming discipline.** Resource ids are + producer-scoped names (OQ-TN-11: collision domain = the producer's + registry). Consumers learn available resources through the + ACL-filtered ops listing (OQ-TN-08) — the spec's + `OperationSpec.description` SHOULD carry a human-readable hint + (e.g. "postgres on the worker's tailnet") since it round-trips + through discovery. +- **No per-connection dynamic targets in base params.** `-D`/SOCKS + composes by tunneling a socks5 connection (OQ-TN-08); a future + multi-endpoint UDP gateway resource would be self-describing + framing inside the channel payload, invisible to params. + +## References + +- Phase 0: OQ-TN-01 (the reframe + survey input), + `docs/research/ssh-socks5-survey.md` §OQ-TN-01 residue +- alkcall ADR-047 (openable ALPNs are operations; `input_schema`), + ADR-049 (typed establishment errors) +- Forward POC: `docs/research/poc-summary.md` (the shape riding 17 + tests); reverse POC: `docs/research/reverse-poc-summary.md` +- OQ-TN-01 (promoted), OQ-TN-11 (naming residue) \ No newline at end of file diff --git a/docs/architecture/decisions/002-alpn-strategy.md b/docs/architecture/decisions/002-alpn-strategy.md new file mode 100644 index 0000000..3d571bf --- /dev/null +++ b/docs/architecture/decisions/002-alpn-strategy.md @@ -0,0 +1,86 @@ +# ADR-002: Single `alk/tunnel` ALPN (Option A) + +## Status + +Accepted (2026-09-07) + +## Context + +alkcall ADR-004: one ALPN per protocol, `alk/` prefix. This crate owns +the `alk/tunnel`-family ALPN(s). The open question (OQ-TN-07) was +whether stream (TCP/unix) and datagram (UDP) tunnels get distinct +ALPNs: + +- **Option A**: single `alk/tunnel` ALPN; the substrate is a `params` + field, and per-substrate data framing (if any) is self-describing + inside the channel. +- **Option B**: `alk/tunnel` (stream) + `alk/tunnel-dgram` + (datagram), so the wire framing differs per ALPN cleanly. + +Mechanically both are cheap: channels' `params` is ALPN-specific, and +the open-handler registry dispatches per ALPN. The cost difference is +consumer-side API shape (two session types vs one with a substrate +enum) and the ALPN namespace (a published string is wire-stable). + +Prior art surveyed in Phase 0 (`ssh-socks5-survey.md`): + +- SSH uses ONE channel mechanism for all forwarding flavors — the + channel type string (`direct-tcpip`, `forwarded-tcpip`, + `direct-streamlocal`) is per-open metadata on one transport, not a + separate transport per flavor. +- SOCKS5 runs CONNECT and UDP ASSOCIATE over one control connection + with a CMD discriminator. +- tun2proxy's udpgw proves datagram framing self-describes over a + stream (length-framed datagrams inside a TCP tunnel). +- With OQ-TN-02 resolved as endpoint-at-open for base UDP resources, + the substrate discriminator in `params` tells the establisher and + pumps which framing to expect — exactly option A's shape. The + discriminator is load-bearing (it selects the framing, ADR-003), + so it must ride the open op; the ALPN carries no substrate + information at all. + +## Decision + +One ALPN: **`alk/tunnel`**. The substrate discriminator in `params` +(ADR-001) selects the data-plane framing; the open op, establisher, +pump handler, and consumer session type are shared across substrates +(one `TunnelSession` with a substrate-shaped data plane, not two +session types). + +- The open op id is `channels/tunnel/sub` (the `channels//sub` + convention of alkcall ADR-047 — the open op is a `Sub`-typed + operation: the reply carries `channel_id` once, the data plane + flows on the channel's `BiStream`). +- The channel's ALPN marker on the open-op spec is `alk/tunnel` + (`ChannelOpenSpec::new("alk/tunnel")`) — the value the + per-ALPN dispatch and the adopted-side manager record for + observability. +- If a future tunnel flavor needs structurally different framing from + byte zero with no params-dependent dispatch, that is a NEW ALPN + decided by a new ADR before its first consumer — never a change to + `alk/tunnel`'s meaning. ALPN strings are wire-stable once + published. + +## Consequences + +- **One session type for consumers** (`TunnelSession` with a + substrate-shaped data plane, ADR-005); no API bifurcation. +- **One registration per producer** regardless of how many substrates + it serves; the resource registry (OQ-TN-11) keys on + `(resource, substrate)`. +- **Framing is params-dependent, not ALPN-dependent.** The establisher + validates `substrate` semantically (the registry lookup); the pump + handler is framing-agnostic (raw pass-through halves — the codec + lives on the consumer/producer edge for UDP, ADR-003). A mismatched + `substrate` value fails at schema/establisher time, loudly. +- **ALPN namespace hygiene:** `alk/tunnel` is published here. The + `alknet/tunnel` spelling in the alknet ADR-071 table predates the + `alknet/` → `alk/` prefix swap (alkcall v0.1.1) — docs and code + must not perpetuate the old prefix. + +## References + +- OQ-TN-07 (promoted, resolved by this ADR) +- `docs/research/ssh-socks5-survey.md` §ALPN-relevant prior art +- alkcall ADR-004 (ALPN convention), ADR-047 (open ops per ALPN) +- AGENTS.md convention 17 (ALPN naming) \ No newline at end of file diff --git a/docs/architecture/decisions/003-codec-and-udp-framing.md b/docs/architecture/decisions/003-codec-and-udp-framing.md new file mode 100644 index 0000000..a8297b3 --- /dev/null +++ b/docs/architecture/decisions/003-codec-and-udp-framing.md @@ -0,0 +1,111 @@ +# ADR-003: Data-Plane Codec — Raw Pass-Through (Stream) / Mandatory Length Framing (UDP) + +## Status + +Accepted (2026-09-07; supersedes the Phase 0 "codec direction" hunch +by making the UDP framing mandatory — reverse POC finding F-2) + +## Context + +The tunnel data plane rides a channels data channel: raw bytes inside +the channel `BiStream` (the channels layer strips its 8-byte header +transparently — alknet ADR-093/ADR-071; AGENTS.md convention 5). The +tunnel protocol owns whatever framing it puts inside the `BiStream`. +A tunnel has one data stream per direction, so there is no +sub-demux key — a 5-byte header like alktty's (with `stream_type`) +would be pure overhead here. + +The Phase 0 direction (`phase-0-findings.md` checklist, 2026-09-06): +raw pass-through for stream substrates (0 B tunnel overhead), +`[len: u16 BE]` per datagram for UDP (2 B/datagram), with the +empty-datagram (`len=0`) semantic left as a Phase 1 residue. + +Two validation passes sharpened this: + +1. **Forward POC (2026-09-06, 17 tests):** the codec round-trips + datagrams split across chunks, batched in one chunk, partial + headers, and empty datagrams (`len=0` is a legal payload, NOT EOF + — EOF is the channels-level `length=0` chunk sentinel; the two + coexist at different layers). The 1400-byte (max ethernet MTU + payload) datagram rides the bounded-buffer path; >65535 is + rejected at frame time. +2. **Reverse POC finding F-2 (2026-09-07, executable-pinned):** in + the RAW pass-through pump shape, an empty UDP datagram is a + zero-byte read from the substrate adapter — `tokio::io::copy` + treats `Ok(0)` as end-of-stream and shuts the pump down. An empty + datagram and the zero-length EOF sentinel are the SAME wire shape + at the pump level. The test pins the non-round-trip so the + conclusion is executable, not anecdotal. + +## Decision + +The data plane inside a `alk/tunnel` channel is: + +- **Stream substrates (`tcp`, `unix`): raw pass-through.** The halves + are the tunnel; zero tunnel-level framing. The only wire overhead + is the channels 8-byte chunk header. A zero-length read on a stream + substrate is genuinely EOF (stream semantics) — no collision. Both + pumps run `alkcall::channels::pump_bidi` (ADR-050) over the raw + halves. +- **Datagram substrate (`udp`): mandatory `[len: u16 BE][datagram]` + framing, applied at the substrate/pump boundary.** One datagram per + length-prefixed frame, on BOTH directions: + - The **establisher** wraps the dialed UDP socket in the framed + adapter BEFORE returning the plan (the plan payload is the + adapter, not a raw socket) — the pump handler stays + substrate-agnostic (ADR-004's invariant; producer.md's placement). + - Consumer side: the `TunnelSession`'s datagram variant does the + same over the adopted halves. + - **`len=0` is a legal empty datagram** (DNS-over-TCP-style + zero-payload probes); it round-trips under the codec (forward + POC). EOF is exclusively the channels-level sentinel; the codec + never emits zero-length reads, so the layers never collide. This + is the load-bearing property F-2 validated from a second angle: + the length prefix makes an empty datagram two bytes — unambiguous + with EOF. +- **MTU discipline:** the codec rejects framing a >65535-byte + datagram (`Oversize` error at frame time, never a wire overflow — + the u16 length field would wrap). A truncated receive (caller + buffer smaller than the datagram) fails loudly at the adapter + level — resolved 2026-09-07 (OQ-TN-13, fail-loud): silent + truncation would corrupt the framing invariants (the same + invariant class F-2 protects). +- **Chunking is transparent.** Datagrams split across channel chunks, + batch into single chunks, and survive partial headers — the + incremental decoder is the only consumer-visible decode path + (forward POC: maximally awkward 7-byte chunk splits verified). + +## Consequences + +- **TCP/unix tunnels have 0 B tunnel overhead** — the SSH + `direct-tcpip` experience (the payload is the bytes). +- **UDP correctness requires the codec** — raw pass-through over UDP + is structurally broken for empty datagrams (F-2). The substrate + discriminator (ADR-001) is what tells both sides to engage the + codec; a consumer bypassing it for UDP is a spec violation, not a + degraded mode. +- **`tokio::io::copy`-shaped pumps fit UDP only through the framed + adapter** (the `UdpHalf` + codec wrapper presents + `AsyncRead`/`AsyncWrite` whose zero-length read never occurs for a + well-formed stream — `len=0` yields a 2-byte wire frame). +- **The BAST document exists** (AGENTS.md convention 12 — the + mandatory framing IS binary framing beyond pass-through, so the + trigger fired): `docs/architecture/bast.md` describes the + `[len: u16 BE]` datagram frame per the BAST meta-schema; the + hand-rolled codec remains the runtime implementation. +- **No mid-stream control frames.** Pump-phase failures are + EOF-shaped by design (alkcall ADR-049 §6); there is no tunnel-level + error frame, no KEEPALIVE (channels transport liveness is + upstream's), no flow-expiry frame (producer-side bookkeeping). + +## References + +- OQ-TN-02 (promoted), OQ-TN-13 (resolved by this ADR's fail-loud + posture) +- Forward POC: `docs/research/poc-summary.md` §3 (codec decision + + sentinel layering + MTU); reverse POC: + `docs/research/reverse-poc-summary.md` §F-2 (the mandate) +- `docs/research/ssh-socks5-survey.md` §UDP (tun2proxy udpgw framing + precedent — no CONN_ID, no FLAGS; boundary framing only) +- alknet ADR-071/093 (channels wire format; the zero-length sentinel) +- alkcall `channels::pump_bidi` (ADR-050) \ No newline at end of file diff --git a/docs/architecture/decisions/004-no-backend-trait.md b/docs/architecture/decisions/004-no-backend-trait.md new file mode 100644 index 0000000..1db5c42 --- /dev/null +++ b/docs/architecture/decisions/004-no-backend-trait.md @@ -0,0 +1,115 @@ +# ADR-004: No TunnelBackend Trait — Halves Functions at the Assembly Layer + +## Status + +Accepted (2026-09-07; resolves the OQ-TN-05 executable input + the hub +re-produce thread) + +## Context + +alktty has a `TtyBackend` trait because backends (local PTY, docker, +SSH) produce complex handles — `TtyHandle` carries +stdin/stdout/stderr/exit-code quadruple + control — and the adapter +pumps them. The Phase 0 question (OQ-TN-05): does alktunnels need a +`TunnelBackend` trait, or is the two-pump handler + feature-gated +substrate modules the whole story? + +The tunnel producer's substrate action is much narrower than a TTY +backend's: + +- **Dial flows:** dial a target, get an `AsyncRead + AsyncWrite` + stream (or a datagram endpoint). A tunnel "handle" is just boxed + halves. +- **Listen flows:** accept on a listener; per accept, the accepted + connection is the same boxed-halves shape. + +The UDP POC gave the executable answer for dial flows (2026-09-06): +`pump_halves` is generic over boxed `(AsyncRead, AsyncWrite)` halves — +TCP contributes `into_split()` halves, UDP contributes the `UdpHalf` +socket adapter, the pump never knows which. "Produce boxed halves for +a resource" is a function, not a trait. + +The one remaining thread was the hub re-produce composition (does a +hub proxy that re-produces a consumed resource need a trait or a +composition helper?). The reverse POC (2026-09-07) resolved the +mechanics: a hub proxy is a consumer + producer composed at the +assembly layer — the reverse POC's hub is exactly this shape (it +consumes a channels connection and produces reverse-tunnel opens +toward the worker); re-exporting (registering an openable for a +resource the hub itself consumes) would add a +`register_openable_with_establisher` whose establisher opens a +consumer channel. Both halves are public-surface operations; no new +mechanism is visible yet. Deferring the helper until a concrete +re-produce consumer exists is the alknet ADR-078-style "genuine +deferral" pattern +pattern (OQ-TN-12). + +## Decision + +**No `TunnelBackend` trait in v1.** The inversion point is the +assembly layer: + +- **The produce function.** The producer's establisher resolves + `(resource, substrate)` → boxed halves + (`Box` ×2 — the `+ Sync` is + the `ChannelPlan` bound, reverse POC F-1) or a framed UDP adapter. + The dial itself is substrate code living in feature-gated backend + modules (`local` feature: real sockets), injected at assembly: + the establisher closure the assembly layer constructs closes over + the substrate dial function. The protocol crate never sees a + socket type. +- **The listen variant** is an establisher shape, not a trait: a + producer that LISTENS (SSH `-R` far-side listener, or the hub's own + exposed port being produced to a third party) registers the same + open op; its establisher pops the next accepted connection from an + assembly-owned listener queue and returns it as the plan payload. + The listener lifecycle (bind, accept loop, cancel) is assembly + code — the protocol never binds (OQ-TN-04). The reverse POC's + worker (dial) and the forward POC's producer (dial) both rode this + shape; the listen establisher is the same plan-flow with a + different halves source. +- **The pump handler is substrate-agnostic by construction:** + `pump_bidi(bidi, t_read, t_write)` over the downcast plan payload — + it cannot know whether the halves came from TCP, UDP, a unix + socket, or an in-process pipe, and must not. +- **Datagram substrates** contribute a framed adapter (ADR-003) + instead of raw halves; the pump shape is unchanged. +- **Re-evaluation trigger:** if a consumer crate demonstrates that + every assembly layer re-implements the same glue (registry + dial + + establisher construction + pump wiring) three or more times, extract + a `TunnelBackend`-shaped trait THEN (the alknet ADR-078 + convergence-test threshold — extract when the shapes have converged + across consumers, not before) + — as an additive layer, not a v1 wire/ABI commitment. Until then + the function-plus-closure shape keeps the crate dependency-light + and wasm-clean. + +## Consequences + +- **The default crate stays wasm-clean** with no socket/platform + deps; `local` (sockets) and future backends (docker, process) are + feature-gated modules implementing dial/listen functions. +- **Assembly layers write more glue than a trait would save** — the + accepted cost, validated twice (both POCs' producers are ~200 lines + including the UDP adapter). +- **Plan payloads are `Send + Sync`** (reverse POC F-1, documented on + alkcall's `ChannelPlan` type in 0.7.0): socket-backed handles carry + `+ Sync` naturally; non-Sync handles (process pipes as boxed trait + objects) need a wrapper. The spec documents the constraint. +- **The listen variant needs no new wire surface** — same open op, + same params, same typed errors (an empty listener queue during + establishment is `resource_shortage`; a closed listener is + `dial_failed`-class). + +## References + +- OQ-TN-05 (promoted), OQ-TN-12 (hub re-produce, deferred), OQ-TN-14 + (unix/stdio placement) +- Forward POC: `docs/research/poc-summary.md` §5 (halves-not-trait); + reverse POC: `docs/research/reverse-poc-summary.md` (the hub shape, + F-1) +- alktty ADR-002 (`TtyBackend` — the contrast case: when a trait IS + warranted), alktty ADR-003 (backend placement — the feature-gate + pattern) +- AGENTS.md conventions 4/7/14 (wasm-clean, substrate-agnostic, + feature flags) \ No newline at end of file diff --git a/docs/architecture/decisions/005-consumer-session-owns-teardown.md b/docs/architecture/decisions/005-consumer-session-owns-teardown.md new file mode 100644 index 0000000..79ac78d --- /dev/null +++ b/docs/architecture/decisions/005-consumer-session-owns-teardown.md @@ -0,0 +1,113 @@ +# ADR-005: The Consumer Session Type Owns Teardown + +## Status + +Accepted (2026-09-07; resolves the reverse POC W3 finding) + +## Context + +A tunnel channel's two ends have asymmetric lifecycle machinery: + +- **The serving side (the side the open op ran on)** is + wrapper-managed: the open wrapper awaits the pump handler's + `JoinHandle`, and its completion triggers channel teardown (drop of + the demux sender = EOF to the handler's read half — alkcall ADR-049 + + review 007 R-02). Out-of-band `channel/close` also tears it down. + Nothing leaks. +- **The adopting side (the side that called the open op and adopted + the channel ID — the consumer)** has NO such machinery: + `ChannelManager::adopt_channel` installs routing state nothing + awaits. The consumer-side pump (`pump_bidi` spawned locally) is + somebody's `JoinHandle`; the adopted channel state is nobody's to + reap. If the assembly layer drops both, the channel entry leaks in + the manager until transport EOF (`clear_all`) — and worse, a + consumer that drops the pump handle mid-flight aborts the pump + without reaping, leaving half-open state on the peer. + +The reverse POC found this by construction (W3, 2026-09-07): the hub +must hold its pump handle and reap (`teardown_channel`) itself — +`ReverseTunnel::join_and_reap`/`close` in the POC. The lifecycle is +assembly-layer by design (OQ-TN-04 — no forced binding means no +protocol-owned listener lifecycle either), but the POC demonstrated +that leaving teardown discipline to each assembly layer's memory is +exactly the kind of gap the spec must close with a type, not a doc +note. + +Related constraint from the same POC pass: one session = one channel += one tunnel (the channel ID is the flow key, OQ-TN-02's resolution); +the session is also the natural owner of the substrate-shaped data +plane (raw halves vs datagram codec, ADR-003). + +## Decision + +The consumer half is a typed session, **`TunnelSession`**, and it owns +its teardown: + +- **Construction:** `TunnelSession::open(client, params)` — calls the + open op (`ChannelClient::open_channel` on the forward path; the + reverse path's `open_reverse_channel` + `adopt_channel` equivalent), + adopts the returned channel ID, splits the channel `BiStream`, and + presents the substrate-shaped data plane: + - **Stream variant:** raw halves (`AsyncRead`/`AsyncWrite`) — the + halves ARE the tunnel. + - **Datagram variant:** `send_datagram`/`recv_datagram` over the + mandatory codec (ADR-003) — boundary-preserving, `len=0` is an + empty datagram, `recv_datagram` returns `None` only on stream EOF. +- **Pump ownership:** the session's `pump_against(accepted_halves)` + (the reverse-flow use) spawns `pump_bidi` and holds the returned + `JoinHandle`. For the pure consumer (no local pump — the session + halves ARE handed to the caller), the session does not spawn; the + caller drives the halves and the session still owns the channel + entry. +- **Teardown API (the point of the ADR):** + - `close(self)` — abort the pump (if session-owned), tear down the + adopted channel (`teardown_channel`), consume the session. The + abort path (ungraceful). + - `join(self)` — await pump completion (both directions finished), + THEN reap the adopted channel, return the `(u64, u64)` copy + counts from `pump_bidi` for observability. The graceful path. + - Dropping the session without either: the `Drop` impl tears the + channel down (the panic-free fallback — never leak the entry). + `Drop` cannot await, so it aborts the pump and calls the + sync `teardown_channel`; this is semantically `close`. +- **Half-open semantics fall out of `pump_bidi`** (ADR-050): one + direction EOFs → the opposite sink shuts down (the EOF sentinel + crosses the mux) → the other pump keeps running until its own EOF. + The session's `join` completes when BOTH pumps finish. Half-close + semantics validated end-to-end (reverse POC W4). +- **Error surface:** a failed open resolves a typed error — + `ChannelOpenError::CallFailed` carrying the wire `CallError`; + branch on `establishment_reason()` for `channel:open_failed`'s + reason (ADR-049 §4). A failed open never yields a session (no + phantom session, mirroring the no-phantom-channel property). +- **`TunnelSession` does not implement `Clone`.** One session = one + channel; aliasing a session would alias its teardown. Multi-channel + consumers hold a `Vec` (or the assembly layer does). + +## Consequences + +- **Assembly layers cannot leak adopted channels** by forgetting to + reap — the type is the discipline (the W3 gap closes structurally). +- **The reverse-flow initiator gets the same session shape** as the + forward consumer: the reverse POC's hub-side `ReverseTunnel` is the + seed; the spec generalizes it so `-L` and `-R` consumers share one + API (role-follows-resource, OQ-TN-03). +- **Observability for free:** `join`'s copy counts surface the + data-plane volumes without extra plumbing. +- **Serving-side parity is upstream's:** the worker's pump handler is + already wrapper-managed (R-02); the session only fixes the adopter + asymmetry. +- **`Drop`-based teardown is best-effort** (abort + sync reap, no + await) — the documented contract; graceful flows call `join` or + `close` explicitly. + +## References + +- Reverse POC: `docs/research/reverse-poc-summary.md` §W3 (the + finding), §W4 (half-close validation) +- OQ-TN-03, OQ-TN-04, OQ-TN-09 (promoted) +- alkcall ADR-049 + review 007 R-02 (wrapper-managed serving side), + ADR-050 (`pump_bidi` + copy counts), ADR-047 §5 (allocation — the + initiator adopts) +- AGENTS.md convention 10 (limits inherited — the session adds no + second bookkeeping layer; it owns exactly one channel's lifecycle) \ No newline at end of file diff --git a/docs/architecture/decisions/006-access-control-posture.md b/docs/architecture/decisions/006-access-control-posture.md new file mode 100644 index 0000000..b86e095 --- /dev/null +++ b/docs/architecture/decisions/006-access-control-posture.md @@ -0,0 +1,102 @@ +# ADR-006: Access-Control Posture — the Open Gate Is the Security Boundary + +## Status + +Accepted (2026-09-07; records the OQ-TN-08 resolution as a decision — +the posture was resolved in Phase 0 but the consumer-visible constants +had no ADR backing) + +## Context + +Tunnels reach local networks: opening a tunnel channel instructs the +producer to dial (or accept for) a target — potentially anything the +producer's host can reach. The open gate is therefore the security +boundary of this crate, in the same sense that SSH's +`AllowTcpForwarding`/`PermitOpen` govern `direct-tcpip` opens. The +question (OQ-TN-08, Phase 0) was how much of that boundary lives in +this crate vs alkcall's op-level ACL vs the assembly layer. + +Phase 0 resolved the conceptual tangle (2026-09-05, the OQ-TN-08 +resolution): at protocol-crate level, a produced resource belongs to +the far side of the connection — the protocol works under that +assumption, so the ACL story is exactly alkcall's existing op-level +ACL. No new policy layer. The hub/overlay mechanism (workers +connecting in, the hub re-exposing resources per ITS policy) is a +downstream, assembly-layer concern (OQ-TN-12's scope). + +What was left implicit — and needs pinning, because it is +consumer-visible and stable-once-published: + +- The scope constant governing tunnel opens (`tunnel:open`) is + ACL-surface: identities will be GRANTED this scope by name once + consumers exist (the same one-way character as ALPN strings, which + ADR-002 treats as wire-stable). +- Whether v1 carries target allowlists / tunnel-specific ownership + machinery. + +## Decision + +1. **The open gate is the boundary, and it is alkcall's ACL.** The + open op's `AccessControl` rides `ChannelCore::register_openable` — + the registry runs the ACL before the establisher. The base gate is + the scope constant **`TUNNEL_OPEN_SCOPE = "tunnel:open"`** + (required scope, exact string — stable once published; consumers + will request grants by it). An identity without the scope is + `FORBIDDEN` before any establisher code runs. +2. **The scope string is wire/ACL-stable** from the first consumer: + renaming is a breaking change to every deployed grant; new gates + are additional scopes on new ops (additive), never a rename. +3. **No target allowlists and no tunnel-specific policy machinery in + v1.** The resource is owned by the far side (the Phase 0 + resolution); a producer that wants finer granularity wires + `OwnershipProvider` checks into ITS establisher — the protocol + provides the seam (the establisher sees the per-call opener + identity, alkcall 0.7.0 CF-006), not the policy. This mirrors + alktty's posture (`TTY_OPEN_SCOPE` + op-level ACL, no + tty-specific allowlist). +4. **The optional ownership seam:** the establisher MAY consult + `OwnershipProvider.owns(id_ref, kind, &id, "tunnel")` (the 4-arg + shape; alknet ADR-050's model) for resource-scoped checks. v1 + does not require it. +5. **Discovery is the ACL-filtered ops listing** (openable channels + are operations, alkcall ADR-047): a consumer learns available + tunnel resources through the existing bidirectional ops listing, + `OperationSpec.description` round-trips through discovery (the + human-readable hint). The live resource-enumeration half (alkcall + OQ-40) stays deferred upstream. +6. **Identity resolution** is alkcall 0.7.0's precedence chain + (CF-005): payload `auth_token` > `ServingConfig.identity` > + transport identity; the establisher/pump handler receive the + per-call opener identity (CF-006). Identity-less dispatch fails + closed (`FORBIDDEN`). +7. **`-D`/dynamic targets need no separate policy hook:** a socks5 + tunnel is an ordinary tunnel to a socks5 resource; whatever ACL + governs that resource governs everything reachable through it, + plus whatever policy the socks5 implementation applies downstream + (target selection is in the socks5 protocol, not params). + +## Consequences + +- **One security knob per producer deployment** (the op-level ACL + + scope grants) instead of a tunnel-specific policy language — the + accepted simplicity; per-resource granularity is an establisher + customization (the seam), not a v1 surface. +- **The scope constant is a compatibility commitment:** treat + `tunnel:open` like the ALPN string. Changing it after consumers + request grants is a breaking ACL change. +- **No allowlist concept** — a wildcard-egress producer + (`0.0.0.0/0`-style egress) is expressed by registering a resource + whose backing is that egress; gating it is the assembly layer's + ownership check at the seam. The protocol stays out of the business. +- **Tunnels reach local networks** — producers MUST treat the gate as + the security boundary; the spec docs state this posture (producer.md). + +## References + +- OQ-TN-08 (promoted, resolved by this ADR) +- alkcall ADR-047 (open ops + ACL wiring), ADR-017 §7 (the + `auth_token` path), ledger CF-005/CF-006 (identity precedence, + per-call opener); alknet ADR-024/ADR-050 (registry layering, + ownership — the alkcall ports ADR-019/ADR-011) +- alktty's `TTY_OPEN_SCOPE` posture (the sibling precedent) +- AGENTS.md convention 13 (access control) \ No newline at end of file diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md new file mode 100644 index 0000000..b5144b2 --- /dev/null +++ b/docs/architecture/open-questions.md @@ -0,0 +1,227 @@ +--- +status: draft +last_updated: 2026-09-07 +--- + +# alktunnels — Open Questions + +Centralized tracker. The Phase 0 OQ ledger (`docs/research/ +phase-0-findings.md`, OQ-TN-01..10) is promoted here with statuses +current as of the Phase 1 spec; new Phase 1 questions continue the +OQ-TN numbering. Format per `docs/sdd_process.md`. + +## Theme 1: Wire format and addressing + +### OQ-TN-01: Target addressing format (open-op `params` shape) + +- **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-01) +- **Status**: resolved — [ADR-001](decisions/001-open-params-layout.md) +- **Priority**: high +- **Resolution**: 2026-09-07 (Phase 1). `params` = self-contained + JSON object: `{ "resource": , "substrate": }`. + The resource identifier is the producer's stable name (a registered + resource id, not an address); substrate is the extensible + discriminator (`tcp` | `udp` | `unix` extensible, additive). The + producer owns the backing; no URL-style general addressing. Reframe + + survey input (ssh-socks5-survey §OQ-TN-01 residue) + both POCs. +- **Cross-references**: OQ-TN-02, OQ-TN-03, ADR-001, ADR-002 + +### OQ-TN-02: Datagram substrates (UDP) — boundary preservation + +- **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-02) +- **Status**: resolved — [ADR-003](decisions/003-codec-and-udp-framing.md) +- **Priority**: high +- **Resolution**: split by path (2026-09-06 survey) + F-2 mandate + (2026-09-07 reverse POC). Base open-op UDP resources: + endpoint-at-open (one channel = one flow). Dynamic/`-D` UDP: + per-datagram addressing inside the tunnel payload, assembly-layer. + Boundary preservation: mandatory `[len: u16 BE]` per-datagram + framing — mandatory for correctness, not cosmetics (F-2; the full + rationale and the executable pin live in ADR-003). Under the codec, + `len=0` is a legal empty datagram (forward POC validated); EOF is + the channels-level sentinel; the layers never collide. +- **Cross-references**: OQ-TN-01, OQ-TN-07, ADR-003 + +### OQ-TN-03: Direction semantics (`-L` / `-R` / dynamic) + +- **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-03) +- **Priority**: high +- **Status**: resolved — hub-owns-the-connection model (2026-09-05) + + the reverse-flow POC (2026-09-07, `reverse-poc-summary.md`). + No protocol-level direction: role follows the resource. `-L`/`-R` + are the same open op with the entry point on different machines; + no advertisement op is needed (the listener is the initiator's own + local resource; a far-side listener is a producer-side listen + establisher over the same open op). `-D` composes at the assembly + layer ("tunnel a socks5 connection"), out of base-crate scope. +- **Cross-references**: OQ-TN-04, OQ-TN-08, ADR-004, ADR-005 + +### OQ-TN-04: No forced local binding + +- **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-04) +- **Priority**: medium +- **Status**: resolved — binding is always assembly-layer and + optional, on either side; the protocol never binds. API surface: + dial flows with no local bind (both POCs), listen flows where the + binding is the producer's establisher (listen variant, ADR-004), + unbound/abstract flows (in-process pipes — the pump is generic over + boxed halves). The concrete produce/consume surface is spec'd in + producer.md / consumer.md (ADR-005). +- **Cross-references**: OQ-TN-03, ADR-004, ADR-005 + +## Theme 2: Crate architecture + +### OQ-TN-05: Backend inversion point — `TunnelBackend` trait? + +- **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-05) +- **Status**: resolved — [ADR-004](decisions/004-no-backend-trait.md) +- **Priority**: medium +- **Resolution**: 2026-09-07 (Phase 1). No trait for substrate access: + "produce boxed halves for a resource" is a function (UDP POC, + OQ-TN-05 executable input). The only remaining thread — the hub + re-produce composition — resolves the same way: a hub proxy is a + consumer + producer composed at the assembly layer (the reverse + POC's hub is exactly this shape minus re-export); a composition + helper is deferred(scope) until a concrete hub-proxy consumer + exists (OQ-TN-12). +- **Cross-references**: OQ-TN-12, ADR-004 + +### OQ-TN-06: The two-pump helper — extract now? + +- **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-06) +- **Priority**: medium +- **Status**: resolved (upstream) — `alkcall::channels::pump_bidi` + landed in alkcall 0.6.0 (ADR-050, review 007 R-03), shape as this + OQ sketched it; this crate is cited as convergence input. alktunnels + consumes the helper; hand-rolling is out. +- **Cross-references**: ADR-005 (consumer pump), ADR-050 (upstream) + +## Theme 3: Protocol surfaces + +### OQ-TN-07: ALPN strategy + +- **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-07) +- **Status**: resolved — [ADR-002](decisions/002-alpn-strategy.md) +- **Priority**: high +- **Resolution**: 2026-09-07 (Phase 1). Option A: single `alk/tunnel` + ALPN; the substrate discriminator in `params` selects the framing. + Prior art (SSH one channel mechanism; SOCKS5 one control + connection; udpgw self-describing framing) gives no reason for the + split; option B's consumer-side API bifurcation buys nothing. +- **Cross-references**: OQ-TN-01, OQ-TN-02, ADR-001, ADR-002 + +### OQ-TN-08: Access control and ownership scope + +- **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-08) +- **Status**: resolved — [ADR-006](decisions/006-access-control-posture.md) + (2026-09-07 — the posture promoted to a decision). The resource is + owned by the far side (alkcall op-level ACL applies as-is; no + tunnel-specific policy layer, no allowlists in v1; the ownership + seam is the establisher seeing the per-call opener identity). + Discovery = the ACL-filtered ops listing (openable channels are + operations, ADR-047); `OperationSpec.description` round-trips + through discovery (review 006 E-02). The live resource-enumeration + half (alkcall OQ-40) stays deferred upstream. Identity: 0.7.0's + precedence chain (token > `ServingConfig.identity` > transport), + per-call opener identity on the open-op hooks (CF-005/CF-006). + The `tunnel:open` scope constant is stable once published (ADR-006 + §2). `-D` = "tunnel a socks5 connection" (target selection in the + socks5 protocol). +- **Cross-references**: OQ-TN-03, ADR-006, producer.md (scope gating) + +### OQ-TN-09: Lifecycle, teardown, and error reporting + +- **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-09) +- **Priority**: high +- **Status**: resolved (superseded by alkcall ADR-049, 0.5.0; narrowed + through 0.6.0/0.7.0) — establishment is the open op's awaited phase; + dial failure is a typed `channel:open_failed` call error (reason ∈ + `dial_failed` / `unknown_resource` / `resource_shortage` / + `handler_error` / `timeout`), never a phantom channel. Residual + resolved by the POCs: half-open semantics = the two-pump contract + (W4 validated); the consumer session owns teardown (W3, ADR-005); + v1 needs no mid-stream control frame (pump-phase failures are + EOF-shaped by design, ADR-049 §6). +- **Cross-references**: ADR-003 (sentinels), ADR-005, alkcall ADR-049 + +### OQ-TN-10: POC scope (Phase 0 closure) + +- **Origin**: [phase-0-findings.md](../research/phase-0-findings.md) (OQ-TN-10) +- **Priority**: low +- **Status**: resolved — UDP POC DONE (2026-09-06, 17 tests), + reverse-flow POC DONE (2026-09-07, 14 tests; re-validated over + alkcall 0.7.0, 16 tests). #3 (unix/stdio) folds into the real + crate's test suite (pump agnosticism confirmed twice); #4 (helper + spike) resolved by alkcall ADR-050. Findings F-1 (plan payloads + `Send + Sync` — documented upstream) and F-2 (UDP codec mandatory) + recorded; W1 filed upstream and RESOLVED (CF-005/CF-006). +- **Cross-references**: ADR-003, OQ-TN-05, OQ-TN-06 + +## Theme 4: Phase 1 residues (new) + +### OQ-TN-11: Resource naming — collision domain and lifecycle of a resource id + +- **Origin**: OQ-TN-01's residue (stable resource name vs target + address), carried into Phase 1 +- **Status**: partially resolved — the collision-domain half is + **resolved by [ADR-001](decisions/001-open-params-layout.md)**: the + domain is the producer's registry (per-producer; two producers on + the same connection may expose the same resource name independently + — the open op resolves against the SERVING side's registry). The + lifecycle half stays open: is a resource registry a producer-side + assembly construct (the POCs' `HashMap` — the leading candidate), + or does it need protocol-level register/rename/retire ops? +- **Priority**: medium +- **Impacts**: producer.md's `ResourceRegistry` shape and the + discovery story (OQ-TN-08); a protocol-level registry op would be a + new wire surface (one-way door). +- **Resolution**: lifecycle half pending — no prior-art pressure + (SSH lists no forwards in-band; discovery is the ops listing); + decide when a consumer needs shared/lifecycle-managed resources. +- **Cross-references**: ADR-001, OQ-TN-08 + +### OQ-TN-12: Hub re-produce composition helper + +- **Origin**: OQ-TN-05's remaining thread (2026-09-05 hub model) +- **Status**: deferred(scope) +- **Priority**: low +- **Blocked on**: a concrete hub-proxy consumer (a crate that + re-produces a consumed tunnel resource) — the reverse POC's hub is + the seed shape (consumer + `register_openable` on the same + connection) but does not exercise re-export; if one materializes, + decide whether a composition helper belongs in this crate or is + assembly-layer wiring of the existing surface. +- **Cross-references**: OQ-TN-05, ADR-004, alkcall ADR-042 (relay) + +### OQ-TN-13: UDP truncation semantics at the codec boundary + +- **Origin**: forward POC Issues Surfaced #3 +- **Status**: resolved — [ADR-003](decisions/003-codec-and-udp-framing.md) + (2026-09-07): **fail-loud at the adapter level.** A receive buffer + smaller than the datagram is an error, never a silent truncation — + a truncated datagram would corrupt the length-framed stream's + framing invariants (the reverse POC's F-2 rationale: the codec's + wire discipline is what keeps empty datagrams unambiguous; silent + truncation is the same class of invariant break). The codec's MTU + bound (65535) makes truncation unreachable for well-formed peers; + failing loud turns a spec violation into a typed error. +- **Priority**: medium +- **Impacts**: the `local` feature's UDP adapter and + `TunnelSession::recv_datagram`'s error surface; consumer-visible + (an error vs silent truncation is a behavioral difference), so the + decision rides an Accepted ADR before the first consumer. +- **Cross-references**: ADR-003, wire.md + +### OQ-TN-14: Unix-socket and stdio substrate placement + +- **Origin**: OQ-TN-10 #3 (the cheap un-POC'd item) +- **Status**: open +- **Priority**: low +- **Impacts**: the `local` feature's module surface (whether unix + dial/listen helpers ship in v1); not a wire/ADR question (the wire + enum already carries `unix` per ADR-001). +- **Resolution**: pending — both POCs confirm pump agnosticism; unix + is cheap (same halves shape as TCP); stdio bridging is a different + lifecycle (no dial — the pipe IS the resource). Fold into the + backend module task; unix leans yes for v1, stdio leans deferred. +- **Cross-references**: ADR-003, ADR-004, overview.md \ No newline at end of file diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md new file mode 100644 index 0000000..cf344d0 --- /dev/null +++ b/docs/architecture/overview.md @@ -0,0 +1,166 @@ +--- +status: draft +last_updated: 2026-09-07 +--- + +# alktunnels — Overview + +Arbitrary bidirectional tunnels over alkcall channels: TCP, UDP, unix +sockets, and other stream or datagram substrates — in the `ssh -L` / +`ssh -D` / `ssh -R` sense. A producer/consumer protocol crate riding +alkcall channels the same way alktty does (`alk/tty` is the sibling +precedent). This document covers the crate's purpose, the resource +model in brief, dependencies, the ALPN, and the module map; component +details are in the sibling documents ([wire.md](wire.md), +[producer.md](producer.md), [consumer.md](consumer.md)). + +## What + +`alktunnels` registers the `alk/tunnel` ALPN's open op on alkcall +channels and provides the typed consumer session: + +- **The producer half** (a `channels/tunnel/sub` open op registered + via `ChannelCore::register_openable_with_establisher`, alkcall + ADR-047/049): the establisher validates params semantically + (registry lookup of `{resource, substrate}` → backing), dials (or + accepts for) the substrate, and returns the dialed handle via + `Establishment::new(plan)`; the pump handler awaits + `alkcall::channels::pump_bidi` over the channel and the substrate + halves (ADR-050, R-02's lifetime contract). +- **The consumer half** (`TunnelSession`, ADR-005): opens tunnel + channels (`ChannelClient::open_channel` on the forward path; the + reverse path's call + adopt equivalent), splits the channel + `BiStream`, presents the substrate-shaped data plane (raw halves or + datagram codec), and owns teardown (close/join/Drop). + +The guiding insight (the OQ-TN-01 reframe): + +> A tunnel is a resource, not an address. `params` identify a produced +> resource + substrate; the producer owns the backing. Rich addressing +> (SOCKS5 ATYP, per-datagram remotes) enters only through the +> `-D`/dynamic composition path — inside the tunnel payload, never in +> the wire. + +## Why + +The crate's purpose is to be the tunnel library for downstream +consumers, the same role alktty plays for terminal sessions: + +- A hub that runs agent workspaces exposes workers' services + (postgres, redis, gitea HTTP) as named resources; a coordinator + opens tunnels to them over alkcall channels — the VPN/SSH-tunnel + service pattern without SSH. +- A worker behind NAT dials the hub and serves its own open ops + (the reverse flow, `ssh -R`); the hub opens tunnel channels toward + it per local accept. Both POCs validated this end-to-end. +- A `-D`-style consumer tunnels a SOCKS5 connection; the socks5 + server (assembly layer) does dynamic target selection inside the + tunnel payload. +- The protocol layer is substrate-agnostic by construction — the + bookkeeping (target addressing, direction, lifecycle) never + hardcodes a substrate (AGENTS.md convention 7). TCP, UDP, unix + sockets, and in-process pipes ride the same open op, pump shape, + and session type. + +## The Resource Model in Brief + +A tunnel channel's life (full detail in [wire.md](wire.md)): + +1. **Open (the establishment phase).** The consumer calls + `channels/tunnel/sub` with `params = {resource, substrate}` + (ADR-001). The producer's establisher — the open op's awaited + phase (alkcall ADR-049) — resolves the resource, dials the + substrate, and returns the handle in the plan. Failure is a typed + `channel:open_failed` (`dial_failed` / `unknown_resource` / + `resource_shortage` / `handler_error` / `timeout`); the channel + never exists on the opener's side afterward. +2. **Pump (the data plane).** Two pumps, one per direction, via + `pump_bidi` (ADR-050): each shuts the opposite sink down on + completion; half-close semantics fall out. Stream substrates ride + raw pass-through (0 B tunnel overhead); UDP rides the mandatory + `[len: u16 BE]` codec (ADR-003, F-2). +3. **Teardown.** The serving side is wrapper-managed (R-02); the + adopting side's `TunnelSession` owns teardown (ADR-005, W3). + EOF sentinels are the channels layer's; the codec never collides + with them. + +Direction is not in the protocol: whoever can reach the target is the +producer; whoever wants the bytes is the consumer; connection +direction (who dialed the transport) is independent of tunnel +direction (OQ-TN-03's hub model). `-L`, `-R`, and `-D` are assembly +shapes over the same open op. + +## Dependencies + +``` +alktunnels (default — wasm-clean) +├── alkcall 0.7.0 (core types, channels: ChannelCore/ChannelClient/manager, +│ pump_bidi, ServingConfig identity seam — CF-005/006) +├── tokio (wasm-clean subset: rt, sync, io-util, macros, time) +├── bytes, futures, serde/serde_json, thiserror, tracing, async-trait +└── (no backend deps — sockets/process live in feature-gated modules) + +alktunnels (local feature) — non-wasm by design +└── adds: tokio/net (TCP dial, UDP bind/connect, unix sockets — unix pending OQ-TN-14) +``` + +Same posture as alktty: the default crate is protocol-only and +wasm-clean (AGENTS.md convention 4); substrate I/O is feature-gated +(ADR-004). The wire format carries no alkcall-internal types +(alktty ADR-006's self-containment rule applies to the tunnel codec — +trivial here, since the codec is a 2-byte length prefix). + +## Feature Gates + +| Feature | Contents | Wasm | +|---------|----------|------| +| *(default)* | params, wire codec, open-op spec, establisher shapes, `TunnelSession` — protocol only | yes | +| `local` | TCP/UDP/unix dial + listen helpers — halves-producing dial/listen functions (ADR-004; unix pending OQ-TN-14) | no | + +## Module Map + +``` +src/ + lib.rs — re-exports; crate docs + params.rs — TunnelParams {resource, substrate} + open-op input schema (ADR-001) + wire.rs — the data-plane codec: frame_datagram / DatagramReader (ADR-003) + producer.rs — the open op: tunnel_open_spec (scope-gated), establisher + shapes (dial + listen), pump handler (pump_bidi), + register_tunnel_openable (ADR-004 shapes) + consumer.rs — TunnelSession: open / adopt / stream_halves / + take_halves / send_datagram / + recv_datagram / pump_against / close / join / Drop (ADR-005) + error.rs — TunnelError + the typed open-error surface (ADR-049 §4) + local/ — the `local` feature module: real socket halves functions +``` + +Public API surface is `lib.rs` re-exports (AGENTS.md convention 16). + +## Design Decisions + +All design decisions are documented as ADRs in [decisions/](decisions/). + +| ADR | Decision | Summary | +|-----|----------|---------| +| [001](decisions/001-open-params-layout.md) | Open-op params: resource + substrate | `{resource, substrate}` — the producer's stable name, not an address; wire-stable from the first consumer | +| [002](decisions/002-alpn-strategy.md) | Single `alk/tunnel` ALPN | Substrate in params selects the framing; no ALPN split, no API bifurcation | +| [003](decisions/003-codec-and-udp-framing.md) | Data-plane codec | Raw pass-through (stream) / mandatory `[len: u16 BE]` (UDP — F-2 mandate); `len=0` is a legal empty datagram | +| [004](decisions/004-no-backend-trait.md) | No TunnelBackend trait | Halves functions at the assembly layer; listen is an establisher shape; trait re-evaluated at the 3-consumer threshold | +| [005](decisions/005-consumer-session-owns-teardown.md) | Consumer session owns teardown | `TunnelSession` with close/join/Drop — the W3 gap closes structurally | +| [006](decisions/006-access-control-posture.md) | Access-control posture | The open gate is the boundary: `tunnel:open` scope (stable once published), op-level ACL, ownership seam, no allowlists in v1 | + +## Open Questions + +Open questions are tracked in [open-questions.md](open-questions.md). +Key questions affecting this document: + +- **OQ-TN-11**: resource naming collision domain + lifecycle (partially resolved) +- **OQ-TN-12**: hub re-produce composition helper (deferred(scope)) +- **OQ-TN-13**: UDP truncation semantics (resolved — fail-loud, ADR-003) +- **OQ-TN-14**: unix/stdio substrate placement (open) + +## References + +- Phase 0: `docs/research/` (findings, both POC summaries, survey) +- Upstream ADRs: see [README.md](README.md) §References +- Sibling: alktty (`/workspace/@alkdev/alktty/docs/architecture/`) \ No newline at end of file diff --git a/docs/architecture/producer.md b/docs/architecture/producer.md new file mode 100644 index 0000000..f828885 --- /dev/null +++ b/docs/architecture/producer.md @@ -0,0 +1,190 @@ +--- +status: draft +last_updated: 2026-09-07 +--- + +# alktunnels — Producer Half + +The producer half: the `channels/tunnel/sub` open op — its spec, its +establisher (dial + listen shapes), its pump handler, and its +registration. The producer is whoever can reach the tunnel target; +connection direction is irrelevant (OQ-TN-03's hub model). The reverse +POC validated the producer running on the connect side of its own +connection (`from_connection_with_serving`). + +Design decisions: [ADR-001](decisions/001-open-params-layout.md) +(params), [ADR-002](decisions/002-alpn-strategy.md) (ALPN), +[ADR-003](decisions/003-codec-and-udp-framing.md) (codec), +[ADR-004](decisions/004-no-backend-trait.md) (no backend trait). This +document is the normative WHAT. + +## The Open-Op Spec + +```text +operation id: channels/tunnel/sub +op type: Sub (reply = one { channel_id } envelope; data plane on the BiStream) +ALPN marker: alk/tunnel (ChannelOpenSpec) +visibility: External +input schema: { resource: string (required), substrate: enum [tcp, udp, unix] (required) } +output schema: { channel_id: integer > 0 } +access control: required_scopes: ["tunnel:open"] +description: SHOULD carry a human-readable hint (round-trips through discovery, OQ-TN-08) +``` + +Registration is via `ChannelCore::register_openable_with_establisher` +(alkcall ADR-049 §5): the registry schema-validates input, runs the +ACL, allocates the channel (connection-owner rule, ADR-047 §5), runs +the establisher as the awaited bounded establishment phase, then +spawns the pump handler. The generic channel ops +(`channel/close`, `channel/control`, `channel/resources/subscribe`) +register separately via `ChannelOperations::register_on` — the +producer (or its assembly layer) MUST register them on the serving +registry so peers can manage channels out-of-band (the recipe alkcall +review 007 §Part B verified — `alkcall/docs/reviews/ +007-establishment-follow-ups-review.md`; R-01 = the `Establishment` +plan payload, R-02 = the `OpenHandler` lifetime contract). + +**Registration timing:** the openable may register AFTER +`from_connection_with_serving` returns (the connect-side serving +posture — the reverse POC's W2): the dispatcher reads through the +shared `Arc` per dispatch. Late registration is a +supported assembly shape, not a race. + +## The Establisher + +The establisher is the open op's awaited, bounded establishment phase +(alkcall ADR-049 §1/§2): validate params semantically, consult +ownership if wired, dial/accept the substrate, and return +`Establishment::new(plan)` — the dialed handle rides the typed-opaque +plan (`ChannelPlan = Arc`; the `+ Sync` bound +constrains plan payloads — socket-backed handles carry it naturally, +reverse POC F-1, documented upstream). On failure: +`EstablishmentError` (the typed reason codes) → the wrapper tears the +channel down and replies `channel:open_failed`. + +**Two shapes** (both return the same plan-flow; ADR-004): + +### 1. Dial establisher (the `-L`/forward shape) + +Registry lookup of `(resource, substrate)` → backing → dial → +halves. The dial function is substrate code injected at assembly +(ADR-004: the establisher closure closes over it); the protocol crate +never sees a socket type. Failure mapping: + +| Producer condition | Typed reason | +|---|---| +| resource not in the registry (or wrong substrate) | `unknown_resource` | +| dial refused / timed out / unreachable | `dial_failed` | +| bind/connect resource exhaustion (fds, ports, slots) | `resource_shortage` | +| establisher-internal failure | `handler_error` | +| the bound exceeded | `timeout` (wrapper-level) | + +### 2. Listen establisher (the producer-side listener variant) + +For a producer whose resource is a LISTENER (SSH `-R` far-side +listener; a hub exposing its own port to a third party): the same +open op, the same params; the establisher pops the next accepted +connection from an assembly-owned listener queue and returns it as +the plan payload. The listener lifecycle (bind, accept loop, cancel) +is assembly code — the protocol never binds (OQ-TN-04). Error +mapping: an empty queue during establishment is `resource_shortage`; +a closed listener is `dial_failed`-class. No new wire surface — the +listen variant is plan-flow with a different halves source. + +**Identity:** the establisher receives the **per-call opener +identity** (the dispatch-resolved identity overlaid onto the +install-time context — alkcall 0.7.0 CF-006). On hub-forwarded opens +it sees the end client, not the hub. `auth` carries the opener's +identity plus the transport-truthful fields (`alpn`, `remote_addr`, +`tls_client_fingerprint`) — never rewritten. Consult it for +ownership-scoped resources if the assembly layer wants +`OwnershipProvider` checks here (OQ-TN-08's optional half). + +**Concurrency:** each open's establisher result flows to ITS pump +handler — concurrent opens of the same resource are independent +(the R-01 plan flow — alkcall review 007 R-01, the +`Establishment::new(plan)` payload; the forward POC's resource-keyed +handoff race is structurally gone, confirmed under concurrency by the +reverse POC). + +## The Pump Handler + +The `OpenHandler` (alkcall ADR-049/050 shape: +`Fn(Value, Option, Connection, AuthContext) -> +JoinHandle<()>`): + +1. Downcast the plan to the concrete handle type + (`Arc::downcast::` — establisher and handler agree + on the type; alkcall never inspects it). +2. `conn.accept_bi()` — the channel's yield-once `BiStream`. +3. `pump_bidi(bidi, handle.read, handle.write)` — **awaited inline** + inside the handler's task. The returned `JoinHandle` MUST track + the data-plane lifetime (R-02): the wrapper awaits it and its + completion triggers channel teardown. **Early return = + teardown-at-birth** — both pumps see instant EOF with zero bytes; + the "spawn-and-forget" shape is the one bug class the R-02 + telemetry (yield-once flag + birth-teardown debug hint) diagnoses + at runtime. Hand-rolling the two-pump shape is out (ADR-050). +4. UDP: the plan payload is the FRAMED adapter (ADR-003) — the pump + is unchanged; the codec lives at the substrate/pump boundary. + +**The handler is substrate-agnostic by construction:** it cannot know +whether the halves came from TCP, UDP, a unix socket, or an +in-process pipe, and must not (AGENTS.md convention 7). + +## Registration API (the spec surface) + +```rust +pub fn register_tunnel_openable( + core: &ChannelCore, // the manager + policy pair + registry: &ResourceRegistry, // assembly-owned: (resource, substrate) -> backing + on_registry: &Arc, // the session's dispatch registry + // (the per-session fork per alkcall ADR-047 §4 + // amendment, 2026-09-03 — NOT the connection + // overlay; Layer-2-registered open ops resolve + // NOT_FOUND, review 004 F-01) + auth: AuthContext, // the install-time context (per-call overlay is upstream's) + dial: DialFn, // substrate halves function (ADR-004; `local` feature or assembly) +) -> Result<(), String> +``` + +- `ResourceRegistry` (assembly construct, OQ-TN-11): maps + `(resource, substrate)` → backing. The POCs' `HashMap` shape is the + candidate; the collision domain is the producer's registry + (ADR-001 — two producers on one connection may expose the same + resource name independently). +- The scope constant `TUNNEL_OPEN_SCOPE = "tunnel:open"`; the open-op + spec builder is public so assembly layers can compose variants + (e.g. extra ACL fields) without forking the wire shape. +- The pump handler factory is public for the same reason. + +## Access control posture + +Tunnels reach local networks — the open gate is the security boundary +(OQ-TN-08): + +- The registry runs `AccessControl` before the establisher (the ACL + rides `register_openable`; free via alkcall). +- `TUNNEL_OPEN_SCOPE` is the base gate; the assembly layer MAY add + resource-scoped checks via the establisher (ownership on the + resource name) — the protocol provides the seam, not the policy. +- No target allowlists, no tunnel-specific policy machinery in v1 — + the far side owns the resource (OQ-TN-08 resolution); discovery is + the ACL-filtered ops listing. + +## Open Questions + +- **OQ-TN-11**: resource naming collision domain + lifecycle + (partially resolved — the per-producer registry is the leading + candidate) +- **OQ-TN-14**: unix/stdio substrate placement (open — backend module + task, not a wire question) + +## References + +- [wire.md](wire.md) (the open op's wire surface), + [overview.md](overview.md) (module map) +- Forward POC `docs/research/poc-summary.md`; reverse POC + `docs/research/reverse-poc-summary.md` (both producer shapes ran) +- alkcall ADR-047/049 (open ops + establishment), ADR-050 (`pump_bidi`), + ledger CF-005/CF-006 (identity precedence + per-call opener) \ No newline at end of file diff --git a/docs/architecture/wire.md b/docs/architecture/wire.md new file mode 100644 index 0000000..34fd2a9 --- /dev/null +++ b/docs/architecture/wire.md @@ -0,0 +1,168 @@ +--- +status: draft +last_updated: 2026-09-07 +--- + +# alktunnels — Wire Format + +The `alk/tunnel` wire surface. Two layers: the **open op** (JSON +params on the channels call plane — structured, schema-validated, +typed errors) and the **data plane** (bytes/datagrams inside the +channel `BiStream` — the codec, ADR-003). The channels layer strips +its own 8-byte header transparently (alknet ADR-093); the tunnel +protocol owns everything inside the `BiStream`. + +Design decisions live in the ADRs ([001](decisions/001-open-params-layout.md), +[002](decisions/002-alpn-strategy.md), [003](decisions/003-codec-and-udp-framing.md)); +this document is the normative WHAT. + +## The Open Op (`channels/tunnel/sub`) + +`Sub`-typed operation (alkcall ADR-047): the reply carries +`{ "channel_id": }` once; the data plane flows on the channel's +`BiStream` — never in the call response stream. + +### Input (params) + +```json +{ + "resource": "postgres-primary", + "substrate": "tcp" +} +``` + +- `resource`: string, required — the producer's stable resource name + (NOT an address; the producer's registry maps it to its backing). +- `substrate`: string, required, enum `["tcp", "udp", "unix"]` — the + extensible discriminator (ADR-001); selects the data-plane framing. +- `input_schema` pins both as required strings; unknown substrate + values fail schema validation (typed error — loud, not silent). +- Extensibility: a new substrate is a new enum value, not a format + change. An older producer rejecting a newer substrate is the SSH + "unknown channel type" posture. + +### Reply / errors + +- Success: `{ "channel_id": }` — the channel is open, the + establishment phase (dial) succeeded, the pump handler is running. +- Failure: a typed `channel:open_failed` call error with + `details: { reason, message }` — `reason ∈ dial_failed` / + `unknown_resource` / `resource_shortage` / `handler_error` / + `timeout` (alkcall ADR-049 §3). The registry's ACL gate runs before + the wrapper (`FORBIDDEN` for a scope-less identity); the 256-channel + cap is `channel:too_many_channels`. **A failed open never returns a + `channel_id`** — no phantom channel (the SSH contract, POC-verified + from the consumer side). +- The scope gate: the open op's `AccessControl.required_scopes` is + `["tunnel:open"]` (the `TUNNEL_OPEN_SCOPE` constant). Caller + identity resolves in the alkcall 0.7.0 precedence order: payload + `auth_token` > `ServingConfig.identity` > transport identity (CF-005); + the establisher and pump handler receive the per-call opener + identity (CF-006). + +## The Data Plane + +One data stream per direction on one channel — the channel ID is the +flow key (no conn-id, no sub-demux). Two codecs by substrate +(ADR-003): + +### Stream substrates (`tcp`, `unix`): raw pass-through + +The halves are the tunnel. Zero tunnel-level framing; the only wire +overhead is the channels 8-byte chunk header. A zero-length read is +genuinely EOF (stream semantics). + +Both pumps run `alkcall::channels::pump_bidi(channel, peer_read, +peer_write)` — the two-pump contract (alknet ADR-078, pinned upstream +ADR-050): when one direction's source EOFs, the pump shuts the +OPPOSITE sink down (the EOF sentinel crosses the mux; the peer sees a +clean half-close), and the helper completes when both pumps finish, +returning `(u64, u64)` copy counts. Copy errors are EOF-shaped +(abrupt close); there is no `Err` state. + +### Datagram substrate (`udp`): mandatory length framing + +Every datagram rides `[len: u16 BE][datagram]` on BOTH directions — +mandatory for correctness (the F-2 rationale lives in ADR-003; the +short form: raw pass-through cannot carry an empty datagram). Under +the codec: + +- **Framing (`frame_datagram`):** `len: u16 BE` + payload. Datagrams + > 65535 bytes are rejected at frame time (`Oversize`) — never a + wire overflow (the u16 length field would wrap). +- **Decoding (`DatagramReader`):** incremental — datagrams split + across channel chunks, batch into single chunks, and survive + partial headers. The decoder buffers state across reads; it is the + only consumer-visible decode path. +- **`len = 0` is a legal empty datagram** (DNS-over-TCP-style + zero-payload probes). EOF is exclusively the channels-level + sentinel (`BiStream` EOF after draining); the codec never emits + zero-length reads, so the layers never collide. A `recv_datagram` + returns `Some(bytes)` per datagram (possibly empty), `None` only on + stream EOF. +- **Chunk-size independence:** datagram boundaries survive chunk + splitting, batching, and any channels-layer re-chunking — verified + end-to-end at 7-byte chunk splits (forward POC). +- **MTU discipline:** 1400-byte datagrams (a conservative MTU-safe + payload, not the theoretical 1472 max) ride the bounded-buffer path; + the 64-parked-chunks bound (alkcall early-arrival cap) is upstream + sizing, not tunnel policy. +- **Truncation:** a receive buffer smaller than the datagram fails + loudly at the adapter level (OQ-TN-13, resolved fail-loud per + ADR-003); silent truncation would corrupt the framing invariants. + +### Sentinels and lifecycle (both codecs) + +- **EOF:** the channels-level zero-length chunk, written by + `pump_bidi`'s shutdown-on-completion or by a channel teardown + (wrapper exit, `channel/close`, transport EOF → demux sender drop). + EOF is per-direction (half-close): one direction EOFs, the other + keeps pumping until its own EOF. +- **No mid-stream control frames.** Pump-phase failures are + EOF-shaped by design (alkcall ADR-049 §6); there is no tunnel-level + error frame, no KEEPALIVE (channels transport liveness is + upstream's), no flow-expiry frame (producer-side bookkeeping, and + idle-expiry mapping is a producer-registry concern, not wire). +- **No establishment frame.** The open op's reply IS the + establishment result (ADR-049); the pre-ADR-049 + "JSON ack before data" hunch is superseded — the wire carries zero + tunnel-level control framing in v1. +- **BAST:** the binary framing (the UDP codec) carries its BAST + document — [bast.md](bast.md) (AGENTS.md convention 12; the + stream pass-through has no binary framing to describe). + +## Normative byte diagrams + +Stream substrate, one direction's wire view (inside the channel +`BiStream`, after the channels layer's 8-byte header per chunk): + +``` + ... +``` + +UDP substrate, one direction's wire view: + +``` +[len:u16 BE][datagram bytes] [len:u16 BE][datagram bytes] ... + len == 0: an empty datagram (legal payload, NOT EOF) + EOF: the channels-level sentinel, outside the codec +``` + +## Open Questions + +- **OQ-TN-13**: resolved (fail-loud, ADR-003) — see the Truncation + bullet +- **OQ-TN-02**: resolved — see the ADR-003 mandate + +## References + +- [ADR-001](decisions/001-open-params-layout.md) (params), + [ADR-002](decisions/002-alpn-strategy.md) (ALPN), + [ADR-003](decisions/003-codec-and-udp-framing.md) (codec), + [ADR-005](decisions/005-consumer-session-owns-teardown.md) (session + teardown) +- Forward POC `docs/research/poc-summary.md` (codec + sentinel + layering, 17 tests); reverse POC + `docs/research/reverse-poc-summary.md` (F-2 mandate, W4 half-close) +- alkcall ADR-047 (open ops), ADR-049 (establishment/typed errors), + ADR-050 (`pump_bidi`); alknet ADR-071/093 (channels wire) \ No newline at end of file