alktunnels/README.md

# alktunnels

Arbitrary bidirectional tunnels over [alkcall](https://crates.io/crates/alkcall)
channels: TCP, UDP, and unix-socket tunnels in the `ssh -L` / `ssh -R`
sense (and the `-D`/SOCKS flavor via composition), without SSH. A
producer/consumer protocol crate riding alkcall channels the same way
[alktty](https://crates.io/crates/alktty) does — no networking, no
transport dependencies; the transport (QUIC, websocket, stdio, ...)
is whatever alkcall runs over.

Downstream crates compose on top of it: wire your substrate's sockets
into the producer's dial/accept/bind closures (the `local` feature
ships reference helpers), and open typed sessions from the consumer
side. Dynamic-target egress (`ssh -W`/SOCKS5 CONNECT shape) rides the
[`direct op`](https://docs.rs/alktunnels/latest/alktunnels/params/constant.OP_TUNNEL_DIRECT.html)
(ADR-007); the composed `-R` flow (bind + peer metadata) rides the
bind-first establisher + the
[`forwarded op`](https://docs.rs/alktunnels/latest/alktunnels/params/constant.OP_TUNNEL_FORWARDED.html)
(ADR-008).

## Quick start

### Producer — register produced resources and serve the open op

```rust
use std::sync::Arc;

use alkcall::channels::operations::ChannelCore;
use alkcall::core::auth::AuthContext;
use alkcall::registry::registration::OperationRegistry;
use alktunnels::params::Substrate;
use alktunnels::producer::{register_tunnel_openable, ResourceRegistry};

// The producer's registry: (resource, substrate) → backing. The
// backing string is whatever your dial closure understands — a
// `host:port`, a socket path, an in-process id. It never rides the
// wire; the consumer only ever names the resource.
let registry = ResourceRegistry::new();
registry.register("postgres-primary", Substrate::Tcp, "127.0.0.1:5432").await;
registry.register("dns", Substrate::Udp, "127.0.0.1:53").await;

// The substrate boundary is a function, not a trait (ADR-004):
// `DialFn` = (substrate, backing) → boxed halves. Behind the `local`
// feature, `alktunnels::local::local_dial()` covers tcp/udp/unix real
// sockets; swap in docker/ssh/in-process closures for anything else.
let dial = alktunnels::local::local_dial();

let op_registry = Arc::new(OperationRegistry::new());
let core = ChannelCore::new(manager, default_policy());

// Registers `channels/tunnel/sub` on the `alk/tunnel` ALPN: the
// registry's ACL (scope `tunnel:open`) gates the open op, the
// establisher dials the backing, and `pump_bidi` bridges the channel
// to the dialed halves. Wire `core` + `op_registry` into your
// channels serving stack (the alkcall recipe) — this call is the
// tunnel-specific step:
register_tunnel_openable(
    &core,
    &registry,
    &op_registry,
    AuthContext::anonymous(b"alk/tunnel"),
    dial,
    None,
)?;
```

### Producer — listen shape (the `-R` far side accepts)

The same open op with a listen establisher: instead of dialing a
target, the establisher pops an already-accepted local connection.

```rust
use alktunnels::producer::{register_tunnel_listen_openable, AcceptQueue, ResourceRegistry};

let queue = AcceptQueue::new();
// Your accept loop owns the listener and feeds the queue; the
// protocol crate never binds (OQ-TN-04). With the `local` feature:
//   let listener = alktunnels::local::bind_tcp("127.0.0.1:8080").await?;
//   tokio::spawn(listener.accept_loop(queue.clone()));
let accept: AcceptFn = /* pops from the queue (or your own source) */;

register_tunnel_listen_openable(&core, &registry, &op_registry, auth, accept)?;
```

### Producer — bind-first listen shape (`-R` with address + peer fidelity)

The same open op with a bind-first establisher: establishment BINDS
(the reply carries the real OS-chosen `bound` address + a correlation
token), then every local accept opens the `channels/tunnel/forwarded`
op toward the listen opener with the real peer address.

```rust
use alktunnels::producer::{
    register_tunnel_bindfirst_openable, BindFirstInjections,
};

let injections = BindFirstInjections {
    // Binds the listener, returns it + the OS-chosen bound address.
    // Behind `local`: alktunnels::local::local_bind_first() (tcp/unix).
    bind,
    // Per accept: yields (peer, halves) from the opaque listener.
    // Behind `local`: alktunnels::local::local_bindfirst_accept_loop().
    accept_loop,
    // Opens the forwarded op on the channel-0 call plane + adopts.
    open_forwarded,
};
register_tunnel_bindfirst_openable(&core, &registry, &op_registry, auth, injections, None)?;

// The forwarded op's opener-side serving half (on the consumer's
// connection): correlate the token, pump against the local socket.
register_tunnel_forwarded_openable(
    &core, &op_registry, auth,
    /* ForwardedHandler: (params, BiStream) -> pump task */,
)?;
```

### Consumer — open a tunnel channel (the `-L` path)

```rust
use alkcall::channels::client::ChannelClient;
use alktunnels::params::{Substrate, TunnelParams};
use alktunnels::TunnelSession;

let client: ChannelClient = /* your alk/channels connection */;

// One session = one channel = one tunnel. A failed open never yields
// a session (typed errors: `unknown_resource`, `dial_failed`,
// `resource_shortage`, `timeout`, `FORBIDDEN`, ...).
let mut session = TunnelSession::open(
    &client,
    TunnelParams { resource: "postgres-primary".into(), substrate: Substrate::Tcp },
).await?;

// Stream substrates (tcp/unix): raw pass-through — the halves ARE
// the tunnel, zero tunnel-level framing.
let (read, write) = session.stream_halves().expect("stream session");
// ... use like any tokio duplex half (or take_halves() for owned) ...

// Datagram substrate (udp): boundary-preserving, framed
// (`[len: u16 BE]`, ADR-003) — an empty datagram is legal and is
// NOT EOF.
let payload = session.recv_datagram().await?; // Some(Bytes) | None on EOF

// Teardown is the session's job (ADR-005): abort + reap, idempotent
// via Drop, or await both pumps with copy counts:
let (client_to_target, target_to_client, reaped) = session.join().await;
```

### Consumer — reverse flow (the `-R` path)

The consumer initiates (it may hold the hub's call surface while the
producer serves); the producer-allocated channel ID is adopted into a
session, then pumped against a local acceptor.

```rust
use alkcall::protocol::connection::CallConnection;
use alktunnels::{open_reverse_channel, TunnelSession};

let hub_call: CallConnection = /* channel-0 call surface */;
let params = TunnelParams { resource: "socks-exit".into(), substrate: Substrate::Tcp };

let channel_id = open_reverse_channel(&hub_call, &params, None).await?;
let session = TunnelSession::adopt(
    &manager,
    channel_id,
    Substrate::Tcp,
    alktunnels::TUNNEL_ALPN,
).await?;

// Hand each locally accepted connection to the session's pump; the
// session keeps teardown ownership (`close` aborts, `join` awaits
// and reaps, `Drop` aborts + reaps).
session = session.pump_against(accepted_local_stream).await;
```

### Consumer — dynamic target (the direct op, `ssh -W`/CONNECT shape)

```rust
use alktunnels::params::SubstrateAddr;
use alktunnels::TunnelSession;

// `tunnel:direct` is a SEPARATE grant from `tunnel:open` (never
// implied by it) — the caller states which capability it holds. A
// malformed target fails the schema gate; an old producer rejects
// the unknown op loudly (NOT_FOUND).
let mut session = TunnelSession::open_direct(
    &client,
    SubstrateAddr::HostPort { host: "example.internal".into(), port: 5432 },
    Substrate::Tcp,
).await?;
```

### Consumer — the listen flow (bind-first reply fields)

```rust
use alktunnels::TunnelListener;

// The same `channels/tunnel/sub` open op — the establisher shape is
// the producer's per-registration choice, invisible here. A bind-first
// producer's reply carries `bound` (the real OS-chosen address) and a
// correlation token; both are `None` for dial/accept-first producers
// or old producers (the lenient, additive posture).
let listener = TunnelListener::open(&client, params).await?;
let bound = listener.bound().cloned(); // Option<SubstrateAddr> — informational, NOT addressing
let token = listener.listen();         // Option<u32> — the forwarded op's correlation key

// Teardown: `close` sends the EOF sentinel (the far accept dispatcher
// tears its listener down), then reaps; `Drop` is the sync best-effort
// form. Forwarded sessions arrive as separate adopted `TunnelSession`s
// on the manager — the assembly correlates them via the token; the
// listener does not own them.
listener.close().await;
```

## Architecture

Two roles, symmetric per the alk convention (avoid "server"/"client":
both sides of a connection can be either; role follows the resource):

| Role | Via `alk/channels` |
|------|--------------------|
| **Producer** (owns the backing: local ports, sockets, in-process services) | [`register_tunnel_openable`](https://docs.rs/alktunnels/latest/alktunnels/producer/fn.register_tunnel_openable.html) / [`register_tunnel_listen_openable`](https://docs.rs/alktunnels/latest/alktunnels/producer/fn.register_tunnel_listen_openable.html) / [`register_tunnel_bindfirst_openable`](https://docs.rs/alktunnels/latest/alktunnels/producer/fn.register_tunnel_bindfirst_openable.html) — the open op (`channels/tunnel/sub`) with a **dial establisher** (target dialing), an **accept-first listen establisher** (pops an already-accepted connection), or a **bind-first establisher** (binds; the reply carries `bound` + token, per-accept forwarded opens carry the peer); the pump handler bridges the channel via [`alkcall::channels::pump_bidi`](https://docs.rs/alkcall/latest/alkcall/channels/pump/fn.pump_bidi.html) |
| **Producer** (dynamic-target egress) | [`register_tunnel_direct_openable`](https://docs.rs/alktunnels/latest/alktunnels/producer/fn.register_tunnel_direct_openable.html) — the direct op (`channels/tunnel/direct`, ADR-007) with its own scope (`tunnel:direct`); the target rides the params, no registry lookup |
| **Consumer** (wants the bytes) | [`TunnelSession`](https://docs.rs/alktunnels/latest/alktunnels/consumer/struct.TunnelSession.html) — [`open`](https://docs.rs/alktunnels/latest/alktunnels/consumer/struct.TunnelSession.html#method.open) (forward) / [`open_direct`](https://docs.rs/alktunnels/latest/alktunnels/consumer/struct.TunnelSession.html#method.open_direct) (dynamic target) / [`adopt`](https://docs.rs/alktunnels/latest/alktunnels/consumer/struct.TunnelSession.html#method.adopt) (reverse), substrate-shaped data planes, teardown ownership; [`TunnelListener`](https://docs.rs/alktunnels/latest/alktunnels/consumer/struct.TunnelListener.html) — the listen flow with the `bound`/`listen` reply fields |

One connection hosts many tunnels (one per channel); sessions are
independent. Both roles are callable from either connection side —
`-L` and `-R` are the same open op with the entry point on different
machines (OQ-TN-03; the reverse POC validated the worker-serves /
hub-initiates shape).

### A tunnel is a resource, not an address (with one additive exception)

The base open-op params are wire-stable (ADR-001) and deliberately
address-free:

```json
{ "resource": "postgres-primary", "substrate": "tcp" }
```

The producer's registry maps the resource to its backing; the consumer
never learns an address. Rich addressing (SOCKS5 ATYP, per-datagram
remotes) enters only through the `-D`/dynamic composition path,
speaking its protocol inside the tunnel payload — never in the wire.

The exception is the direct op (ADR-007) — a *different capability*
on its own op + scope, so the ACL separation is structural:

```json
{ "substrate": "tcp", "target": { "host": "10.0.0.7", "port": 5432 } }
```

Scope-gated separately (`tunnel:direct` ≠ `tunnel:open` ≠
`tunnel:forwarded` — never implied by each other); dial-only (no
listen/direct-BIND form); the same `DialFn` the base op uses. The
composed `-R` metadata flow rides the forwarded op
(`channels/tunnel/forwarded`, `{listen, substrate, peer}` — the peer
is producer-asserted metadata, not transport truth).

### Data plane

- **Stream substrates (`tcp`, `unix`)**: raw pass-through — the
  channel's read/write halves are the tunnel; zero tunnel-level
  framing, zero-length reads are EOF (ADR-003).
- **Datagram substrate (`udp`)**: mandatory `[len: u16 BE]`
  per-datagram framing (boundary preservation; `len = 0` is a legal
  empty datagram, never EOF — the F-2 layering; >65535 fails at frame
  time). The machine-readable contract is the BAST document at
  `docs/architecture/bast.md`.
- **Sentinels and backpressure** are alkcall channels invariants
  (zero-length-chunk EOF, bounded buffers, the 256-channel cap) —
  this crate re-derives none of them.

The `local` feature's [`UdpHalf`](https://docs.rs/alktunnels/latest/alktunnels/local/struct.UdpHalf.html)
composes the codec at the substrate boundary: the pump-facing side
speaks framed channel bytes, the target socket sees payload-only
datagrams. Truncation fails loud, never silent (OQ-TN-13).

### WASM target

The default crate (no features) compiles to `wasm32-unknown-unknown`
— the protocol layer for a sandboxed TS/Python adapter. The `local`
feature (real sockets) needs a real OS and is non-wasm by design;
enabling it on wasm is a compile error.

## Version 0.2 — the graduation

0.2 rides alkcall 0.8.0 and graduates the proxy surface
(`docs/research/tunnels-graduation.md`, ADRs 007/008):

- **The direct op** (`channels/tunnel/direct`, scope `tunnel:direct`)
  — dynamic-target egress; `TunnelSession::open_direct`.
- **The bind-first establisher + the forwarded op**
  (`channels/tunnel/forwarded`, scope `tunnel:forwarded`) — the
  composed `-R` metadata flow: `bound` + correlation token in the
  listen reply, `{listen, substrate, peer}` per accept;
  `TunnelListener` + `parse_listen_reply`/`listen_reply_fields`.
- **alkcall 0.7.1 → 0.8.0** — `Establishment::with_reply_field` +
  `open_channel_with_reply` (the bind-first carriers) and
  flavor-form op ids in discovery (hub relay wraps them as channels);
  bonus: in-tree `ChannelRelay`/`HubLegTemplate` (alkcall ADR-051).

0.2.1 is the hardening release: the `fuzz/` workspace (three
cargo-fuzz targets over the datagram codec, the params parsers, and a
stateful session-pump target; corpora replay as plain `cargo test`) —
`docs/research/fuzzing.md` — and the alkcall bump 0.8.0 → 0.8.1
(duplicate adopt/open no longer destroys the live channel — relevant
to the reverse-path adopt flow). Two additive diagnostics:
`TunnelSession::datagram_backlog()` and
`DatagramReader::retained_bytes()`. No API break, no wire change.

0.3.0 rides the alkcall bump 0.8.1 → 0.9.0 (the stub-sweep release —
the four registered-but-always-failing stubs became real
implementations; none were on this crate's paths). No alktunnels API
change, no wire change; minor rather than patch because the
semver-incompatible dependency bump + this crate's alkcall type
re-exports make it semver-visible downstream.

0.4.0 rides the alkcall bump 0.9.0 → 0.10.0 (the gateway
ownership-seam release — the loud ownership-wiring default on the
`GatewayDispatch` spine, behind the opt-in `gateway` feature; none of
it is on this crate's paths, which serve via
`ChannelCore::register_openable*` dispatch). No alktunnels API
  change, no wire change; same minor-not-patch reasoning as 0.3.0.

0.4.1 rides the alkcall bump 0.10.0 → 0.10.1 (the adoption-window
ordering fix — a chunk routed inside the `adopt_channel` install→drain
window can no longer overtake earlier-parked early arrivals). No
alktunnels API change, no wire change; patch because the dependency
bump is backwards-compatible.

## Features

| Feature | Contents | wasm |
|---------|----------|------|
| *(default)* | params, wire codec, open-op specs (`sub`/`direct`/`forwarded`), establishers (dial + accept-first listen + bind-first), `TunnelSession`, `TunnelListener`, registration helpers — protocol only | yes |
| `local` | [`dial_tcp`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.dial_tcp.html) / [`bind_tcp`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.bind_tcp.html) / [`connect_udp`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.connect_udp.html) (the framed `UdpHalf`) / [`dial_unix`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.dial_unix.html) / [`local_dial`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.local_dial.html), the TCP accept loop; the bind-first halves: [`bind_tcp_first`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.bind_tcp_first.html) / [`bind_unix_first`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.bind_unix_first.html) + [`local_bind_first`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.local_bind_first.html) + [`local_bindfirst_accept_loop`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.local_bindfirst_accept_loop.html) | no |

Stdio is intentionally out of scope — a spawned process's stdio is
alktty's pipe mode (exit codes + signals are tty semantics); remote
command execution composes via alktty (OQ-TN-14).

## Verification

```bash
cargo test                                    # default crate (wasm-clean)
cargo test --features local                   # + real-socket suites
cargo clippy --all-targets -- -D warnings
cargo fmt --check
cargo check --target wasm32-unknown-unknown   # the wasm-clean guard
```

## License

MIT OR Apache-2.0