diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..83b1c49 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,52 @@ +# Changelog + +All notable changes to this crate are documented here. The format is +based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and +this crate adheres to [Semantic Versioning](https://semver.org/). + +## [0.1.0] - 2026-09-10 + +The initial release: the v1 tunnel protocol implementation over +alkcall 0.7.x channels, spec-complete (Phase 1) and implementation- +complete (Phase 2, review 001 findings remediated). + +### Added + +- **Wire surface** (one-way-door stable from this release): the + `channels/tunnel/sub` open op on the `alk/tunnel` ALPN; params + `{resource, substrate}` (`deny_unknown_fields`; ADR-001); the + `tunnel:open` scope gate (ADR-006). +- **Data-plane codec** (ADR-003): raw pass-through for stream + substrates (`tcp`, `unix`); mandatory `[len: u16 BE]` boundary- + preserving framing for `udp` (`len = 0` a legal empty datagram, + never EOF; oversize rejected at frame time; truncation fails loud + — OQ-TN-13). The BAST contract: `docs/architecture/bast.md`. +- **Producer half** (ADR-004): `ResourceRegistry`, + `tunnel_establisher` (dial shape) and `listen_establisher` (the + `-R` far-side accept shape) over alkcall's establishment surface + (ADR-049), `make_tunnel_pump_handler` (the `pump_bidi` two-pump + contract, alkcall ADR-050), and the + `register_tunnel_openable` / `register_tunnel_listen_openable` + registration helpers. +- **Consumer half** (ADR-005): `TunnelSession` — `open` (forward) / + `adopt` (reverse), substrate-shaped data planes (`stream_halves`, + `take_halves`, `send_datagram`/`recv_datagram`), pump ownership + (`pump_against`), teardown ownership (`close`/`join`/`Drop`); + `open_reverse_channel` for the hub-call path. +- **`local` feature** (non-wasm): `dial_tcp`, `bind_tcp` + + `TcpListenerHalves` accept loop, `connect_udp` wrapped in the + FRAMED `UdpHalf` adapter (the codec composes at the substrate + boundary — review 001 U-1), `dial_unix`, and the all-substrate + `local_dial` closure. Stdio intentionally out of scope (alktty's + pipe mode owns process stdio — OQ-TN-14). +- **Identity seam** (alkcall 0.7.0 CF-005/CF-006): payload-token > + `ServingConfig.identity` > transport identity precedence proven + end-to-end; the per-call opener identity reaches the establisher. +- **Test suite**: 71 tests default / 91 with `local`, including the + review 001 gates (real-socket UDP framing probes, the + `AcceptQueue` lost-wakeup regression, the identity-precedence + matrix). Wasm-clean default crate verified (`wasm32-unknown-unknown` + check + clippy). + +[Unreleased]: https://git.alk.dev/alkdev/alktunnels/compare/v0.1.0...HEAD +[0.1.0]: https://git.alk.dev/alkdev/alktunnels/releases/tag/v0.1.0 \ No newline at end of file diff --git a/Cargo.toml b/Cargo.toml index 5ddf374..5b893ae 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -9,7 +9,7 @@ repository = "https://git.alk.dev/alkdev/alktunnels" readme = "README.md" keywords = ["tunnel", "proxy", "channels", "alkcall", "network"] categories = ["network-programming", "asynchronous"] -exclude = [".opencode/", "AGENTS.md", "docs/reviews/", "docs/research/", "docs/plans/", "docs/sdd_process.md"] +exclude = [".opencode/", "AGENTS.md", "docs/reviews/", "docs/research/", "docs/plans/", "tasks/", "docs/sdd_process.md"] [lib] name = "alktunnels" diff --git a/README.md b/README.md index c5c894a..cfc0a80 100644 --- a/README.md +++ b/README.md @@ -1,49 +1,215 @@ # alktunnels Arbitrary bidirectional tunnels over [alkcall](https://crates.io/crates/alkcall) -channels: TCP, UDP, unix sockets, and other stream or datagram -substrates — in the `ssh -L` / `ssh -D` / `ssh -R` sense, without SSH. +channels: TCP, UDP, and unix-socket tunnels in the `ssh -L` / `ssh -R` +sense (and the `-D`/SOCKS flavor via composition), without SSH. A +producer/consumer protocol crate riding alkcall channels the same way +[alktty](https://crates.io/crates/alktty) does — no networking, no +transport dependencies; the transport (QUIC, websocket, stdio, ...) +is whatever alkcall runs over. -A producer/consumer protocol crate riding alkcall channels the same -way [alktty](https://crates.io/crates/alktty) does (`alk/tty` is the -sibling precedent): the producer half registers the `alk/tunnel` open -op and pumps bytes between the channel and the substrate; the consumer -half is the typed session (`TunnelSession`) that opens tunnel channels -and owns teardown. +Downstream crates compose on top of it: wire your substrate's sockets +into the producer's dial/accept closures (the `local` feature ships +reference helpers), and open typed sessions from the consumer side. -## A tunnel is a resource, not an address +## Quick start -Open-op params identify a produced resource + substrate: +### Producer — register produced resources and serve the open op + +```rust +use std::sync::Arc; + +use alkcall::channels::operations::ChannelCore; +use alkcall::core::auth::AuthContext; +use alkcall::registry::registration::OperationRegistry; +use alktunnels::params::Substrate; +use alktunnels::producer::{register_tunnel_openable, ResourceRegistry}; + +// The producer's registry: (resource, substrate) → backing. The +// backing string is whatever your dial closure understands — a +// `host:port`, a socket path, an in-process id. It never rides the +// wire; the consumer only ever names the resource. +let registry = ResourceRegistry::new(); +registry.register("postgres-primary", Substrate::Tcp, "127.0.0.1:5432").await; +registry.register("dns", Substrate::Udp, "127.0.0.1:53").await; + +// The substrate boundary is a function, not a trait (ADR-004): +// `DialFn` = (substrate, backing) → boxed halves. Behind the `local` +// feature, `alktunnels::local::local_dial()` covers tcp/udp/unix real +// sockets; swap in docker/ssh/in-process closures for anything else. +let dial = alktunnels::local::local_dial(); + +let op_registry = Arc::new(OperationRegistry::new()); +let core = ChannelCore::new(manager, default_policy()); + +// Registers `channels/tunnel/sub` on the `alk/tunnel` ALPN: the +// registry's ACL (scope `tunnel:open`) gates the open op, the +// establisher dials the backing, and `pump_bidi` bridges the channel +// to the dialed halves. Wire `core` + `op_registry` into your +// channels serving stack (the alkcall recipe) — this call is the +// tunnel-specific step: +register_tunnel_openable( + &core, + ®istry, + &op_registry, + AuthContext::anonymous(b"alk/tunnel"), + dial, + None, +)?; +``` + +### Producer — listen shape (the `-R` far side accepts) + +The same open op with a listen establisher: instead of dialing a +target, the establisher pops an already-accepted local connection. + +```rust +use alktunnels::producer::{register_tunnel_listen_openable, AcceptQueue, ResourceRegistry}; + +let queue = AcceptQueue::new(); +// Your accept loop owns the listener and feeds the queue; the +// protocol crate never binds (OQ-TN-04). With the `local` feature: +// let listener = alktunnels::local::bind_tcp("127.0.0.1:8080").await?; +// tokio::spawn(listener.accept_loop(queue.clone())); +let accept: AcceptFn = /* pops from the queue (or your own source) */; + +register_tunnel_listen_openable(&core, ®istry, &op_registry, auth, accept)?; +``` + +### Consumer — open a tunnel channel (the `-L` path) + +```rust +use alkcall::channels::client::ChannelClient; +use alktunnels::params::{Substrate, TunnelParams}; +use alktunnels::TunnelSession; + +let client: ChannelClient = /* your alk/channels connection */; + +// One session = one channel = one tunnel. A failed open never yields +// a session (typed errors: `unknown_resource`, `dial_failed`, +// `resource_shortage`, `timeout`, `FORBIDDEN`, ...). +let mut session = TunnelSession::open( + &client, + TunnelParams { resource: "postgres-primary".into(), substrate: Substrate::Tcp }, +).await?; + +// Stream substrates (tcp/unix): raw pass-through — the halves ARE +// the tunnel, zero tunnel-level framing. +let (read, write) = session.stream_halves().expect("stream session"); +// ... use like any tokio duplex half (or take_halves() for owned) ... + +// Datagram substrate (udp): boundary-preserving, framed +// (`[len: u16 BE]`, ADR-003) — an empty datagram is legal and is +// NOT EOF. +let payload = session.recv_datagram().await?; // Some(Bytes) | None on EOF + +// Teardown is the session's job (ADR-005): abort + reap, idempotent +// via Drop, or await both pumps with copy counts: +let (client_to_target, target_to_client, reaped) = session.join().await; +``` + +### Consumer — reverse flow (the `-R` path) + +The consumer initiates (it may hold the hub's call surface while the +producer serves); the producer-allocated channel ID is adopted into a +session, then pumped against a local acceptor. + +```rust +use alkcall::protocol::connection::CallConnection; +use alktunnels::{open_reverse_channel, TunnelSession}; + +let hub_call: CallConnection = /* channel-0 call surface */; +let params = TunnelParams { resource: "socks-exit".into(), substrate: Substrate::Tcp }; + +let channel_id = open_reverse_channel(&hub_call, ¶ms, None).await?; +let session = TunnelSession::adopt( + &manager, + channel_id, + Substrate::Tcp, + alktunnels::TUNNEL_ALPN, +).await?; + +// Hand each locally accepted connection to the session's pump; the +// session keeps teardown ownership (`close` aborts, `join` awaits +// and reaps, `Drop` aborts + reaps). +session = session.pump_against(accepted_local_stream).await; +``` + +## Architecture + +Two roles, symmetric per the alk convention (avoid "server"/"client": +both sides of a connection can be either; role follows the resource): + +| Role | Via `alk/channels` | +|------|--------------------| +| **Producer** (owns the backing: local ports, sockets, in-process services) | [`register_tunnel_openable`](https://docs.rs/alktunnels/latest/alktunnels/producer/fn.register_tunnel_openable.html) / [`register_tunnel_listen_openable`](https://docs.rs/alktunnels/latest/alktunnels/producer/fn.register_tunnel_listen_openable.html) — the open op (`channels/tunnel/sub`) with a **dial establisher** (target dialing) or a **listen establisher** (pops an accepted connection); the pump handler bridges the channel via [`alkcall::channels::pump_bidi`](https://docs.rs/alkcall/latest/alkcall/channels/pump/fn.pump_bidi.html) | +| **Consumer** (wants the bytes) | [`TunnelSession`](https://docs.rs/alktunnels/latest/alktunnels/consumer/struct.TunnelSession.html) — [`open`](https://docs.rs/alktunnels/latest/alktunnels/consumer/struct.TunnelSession.html#method.open) (forward) / [`adopt`](https://docs.rs/alktunnels/latest/alktunnels/consumer/struct.TunnelSession.html#method.adopt) (reverse), substrate-shaped data planes, teardown ownership | + +One connection hosts many tunnels (one per channel); sessions are +independent. Both roles are callable from either connection side — +`-L` and `-R` are the same open op with the entry point on different +machines (OQ-TN-03; the reverse POC validated the worker-serves / +hub-initiates shape). + +### A tunnel is a resource, not an address + +The open-op params are wire-stable (ADR-001) and deliberately +address-free: ```json { "resource": "postgres-primary", "substrate": "tcp" } ``` -The producer owns the backing — the consumer never learns an address. -Rich addressing (SOCKS5 ATYP, per-datagram remotes) enters only -through the `-D`/dynamic composition path, inside the tunnel payload, -never in the wire (ADR-001). +The producer's registry maps the resource to its backing; the consumer +never learns an address. Rich addressing (SOCKS5 ATYP, per-datagram +remotes) enters only through the `-D`/dynamic composition path, +speaking its protocol inside the tunnel payload — never in the wire. -- Wire format: `docs/architecture/wire.md` (+ the BAST document at - `docs/architecture/bast.md`) -- Open-op params: ADR-001; codec: ADR-003; ALPN: ADR-002 -- Producer shapes: ADR-004; consumer session: ADR-005; ACL: ADR-006 -- The full spec set: `docs/architecture/` (overview, wire, producer, - consumer, six ADRs, OQ tracker) +### Data plane -## Status +- **Stream substrates (`tcp`, `unix`)**: raw pass-through — the + channel's read/write halves are the tunnel; zero tunnel-level + framing, zero-length reads are EOF (ADR-003). +- **Datagram substrate (`udp`)**: mandatory `[len: u16 BE]` + per-datagram framing (boundary preservation; `len = 0` is a legal + empty datagram, never EOF — the F-2 layering; >65535 fails at frame + time). The machine-readable contract is the BAST document at + `docs/architecture/bast.md`. +- **Sentinels and backpressure** are alkcall channels invariants + (zero-length-chunk EOF, bounded buffers, the 256-channel cap) — + this crate re-derives none of them. -v0.1.0 — the v1 protocol implementation, Phase-2 complete (spec: -`docs/architecture/`; reviews: `docs/reviews/`). The wire surface -(params, ALPN, codec) is one-way-door stable from this point; the -API surface becomes ABI-stable at the first external consumer. +The `local` feature's [`UdpHalf`](https://docs.rs/alktunnels/latest/alktunnels/local/struct.UdpHalf.html) +composes the codec at the substrate boundary: the pump-facing side +speaks framed channel bytes, the target socket sees payload-only +datagrams. Truncation fails loud, never silent (OQ-TN-13). + +### WASM target + +The default crate (no features) compiles to `wasm32-unknown-unknown` +— the protocol layer for a sandboxed TS/Python adapter. The `local` +feature (real sockets) needs a real OS and is non-wasm by design; +enabling it on wasm is a compile error. ## Features | Feature | Contents | wasm | |---------|----------|------| -| *(default)* | params, wire codec, open-op spec, establisher shapes, `TunnelSession` — protocol only | yes | -| `local` | TCP/UDP/unix dial + listen helpers (real sockets) | no | +| *(default)* | params, wire codec, open-op spec, establishers (dial + listen), `TunnelSession`, registration helpers — protocol only | yes | +| `local` | [`dial_tcp`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.dial_tcp.html) / [`bind_tcp`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.bind_tcp.html) / [`connect_udp`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.connect_udp.html) (the framed `UdpHalf`) / [`dial_unix`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.dial_unix.html) / [`local_dial`](https://docs.rs/alktunnels/latest/alktunnels/local/fn.local_dial.html), the TCP accept loop | no | + +Stdio is intentionally out of scope — a spawned process's stdio is +alktty's pipe mode (exit codes + signals are tty semantics); remote +command execution composes via alktty (OQ-TN-14). + +## Status + +v0.1.0 — the v1 protocol implementation, Phase-2 complete. Spec: +`docs/architecture/` (overview, wire, producer, consumer, BAST, ADRs +001–006, the OQ tracker); review 001 (spec-conformance, pre-publish +gate): `docs/reviews/`. The wire surface (params, ALPN, codec, open-op +scope) is one-way-door stable from this point; the API surface becomes +ABI-stable at the first external consumer. ## Verification diff --git a/docs/architecture/README.md b/docs/architecture/README.md index daba6c5..3ebb293 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -1,6 +1,6 @@ --- -status: draft -last_updated: 2026-09-07 +status: implemented +last_updated: 2026-09-10 --- # alktunnels — Architecture diff --git a/docs/architecture/bast.md b/docs/architecture/bast.md index 3d89d5e..e63c894 100644 --- a/docs/architecture/bast.md +++ b/docs/architecture/bast.md @@ -1,6 +1,6 @@ --- -status: draft -last_updated: 2026-09-07 +status: implemented +last_updated: 2026-09-10 --- # alktunnels — BAST Document for the `alk/tunnel` Wire Format diff --git a/docs/architecture/consumer.md b/docs/architecture/consumer.md index 902950b..7af7e36 100644 --- a/docs/architecture/consumer.md +++ b/docs/architecture/consumer.md @@ -1,6 +1,6 @@ --- -status: draft -last_updated: 2026-09-07 +status: implemented +last_updated: 2026-09-10 --- # alktunnels — Consumer Half diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md index b6705cc..ad43f2b 100644 --- a/docs/architecture/open-questions.md +++ b/docs/architecture/open-questions.md @@ -1,6 +1,6 @@ --- -status: draft -last_updated: 2026-09-07 +status: implemented +last_updated: 2026-09-10 --- # alktunnels — Open Questions diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index ab2c265..8603418 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -1,6 +1,6 @@ --- -status: draft -last_updated: 2026-09-07 +status: implemented +last_updated: 2026-09-10 --- # alktunnels — Overview diff --git a/docs/architecture/producer.md b/docs/architecture/producer.md index bb4c758..db14e28 100644 --- a/docs/architecture/producer.md +++ b/docs/architecture/producer.md @@ -1,6 +1,6 @@ --- -status: draft -last_updated: 2026-09-07 +status: implemented +last_updated: 2026-09-10 --- # alktunnels — Producer Half diff --git a/docs/architecture/wire.md b/docs/architecture/wire.md index 0b9597e..7e0147d 100644 --- a/docs/architecture/wire.md +++ b/docs/architecture/wire.md @@ -1,6 +1,6 @@ --- -status: draft -last_updated: 2026-09-07 +status: implemented +last_updated: 2026-09-10 --- # alktunnels — Wire Format