diff --git a/docs/research/phase-0-findings.md b/docs/research/phase-0-findings.md index 4666a4b..3598040 100644 --- a/docs/research/phase-0-findings.md +++ b/docs/research/phase-0-findings.md @@ -416,6 +416,19 @@ re-chunked arbitrarily)? (OQ-TN-10 #1) remains +EV for the channels-layer fit (chunk-size vs datagram-size, MTU vs bounded buffers, idle expiry). +**Mandate strengthened 2026-09-07 (reverse POC finding F-2):** the +length-framed codec is not just boundary preservation — it is +*mandatory for correctness* on UDP. In the raw pass-through pump +shape, an empty datagram is a zero-byte read from the substrate +adapter — indistinguishable from EOF (`tokio::io::copy` treats `Ok(0)` +as end-of-stream and shuts the pump down). An empty datagram and the +zero-length EOF sentinel are the same wire shape at the pump level; +the `[len: u16 BE]` prefix makes an empty datagram two bytes, +unambiguous. UDP rides the codec, never raw pass-through; raw +pass-through stays stream-substrate-only (where a zero-byte read is +genuinely EOF). Validated from a second angle by the reverse-flow POC +(`reverse-poc-summary.md` §F-2, pinned by an executable test). + ### OQ-TN-03: Direction semantics (`-L` / `-R` / dynamic) **Status: resolved 2026-09-05** by the hub-owns-the-connection model @@ -460,9 +473,17 @@ connection. The open questions: "SOCKS5 server implemented over alktunnels dial primitives" in a separate crate. Keep or cut for v1? -**Status:** open — needs architecture decision. Half-answer (hunch): `-L` -is the channel/open handler; `-R` needs a small advertisement/lifecycle -surface; `-D` composes on top and is out of scope for the base crate. +**Status:** resolved 2026-09-05 by the hub-owns-the-connection model; +the last open thread (the `-R` advertisement/lifecycle surface) closed +2026-09-07 by the reverse-flow POC (`reverse-poc-summary.md`): **no +advertisement op is needed.** The listener is the initiating side's +own local resource — it binds (assembly layer, OQ-TN-04) and opens +tunnel channels toward the serving side per accept; a far-side +listener ("expose a port on your end") is a producer-side listen +establisher over the same open op, not a new mechanism. The hunch +above is superseded: `-L` and `-R` are the same open op with the +entry point on different machines; `-D` composes on top and stays out +of scope for the base crate. ### OQ-TN-04: No forced local binding @@ -744,6 +765,19 @@ Phase 0 may need (in rough priority order, per the SDD process's Template available: SSH's `tcpip-forward` global-request registration → per-accept `forwarded-tcpip` opens → cancel (`ssh-socks5-survey.md` §RFC 4254 §7.1). + + **Done 2026-09-07** — see `reverse-poc-summary.md`. 14 tests pass + over alkcall 0.6.0; the review 007 non-finding trace confirmed by + execution (`from_connection_with_serving` + `register_on` + + serving-side allocation). The advertisement thread of OQ-TN-03 + closes: no advertisement op needed — the listener is the hub's own + local resource; a far-side listener is a producer-side listen + establisher over the same open op. Findings: F-1 (plan payloads + must be `Send + Sync`), F-2 (UDP raw pass-through cannot carry + empty datagrams — the length-framed codec is mandatory, second + validation of the codec decision). W1 (connect-side serving path + resolves identity only via payload `auth_token`) filed for the + alkcall ledger. 3. **Unix socket + stdio bridge POC** — cheap; validates "substrate agnostic" beyond IP substrates. 4. **Two-pump helper extraction spike** — OQ-TN-06, only after 1–3. @@ -834,13 +868,23 @@ Candidate reading for the research specialist (to be expanded): `[len:u16 BE]` per datagram for UDP (2 B); no 5-byte header (a tunnel has one data stream per direction — no sub-demux key needed). Residual for the Phase 1 ADR: empty-datagram - (`len=0`) semantics. Feeds OQ-TN-07 (option A strengthened: + (`len=0`) semantics — **strengthened 2026-09-07 (F-2): the + codec is mandatory for UDP** (empty datagram = zero-byte read + = EOF collision in raw pass-through); the empty-datagram + semantic under the codec is `len=0` = a legal empty datagram + (forward POC validated). Feeds OQ-TN-07 (option A strengthened: substrate discriminator in params selects the framing) and the OQ-TN-05 trait shape - [ ] Decision input: backend trait vs no-trait (OQ-TN-05), now including the hub re-produce composition question -- [ ] Targeted POC(s) run + summary (OQ-TN-10) — UDP POC DONE - (2026-09-06, `poc-summary.md`); reverse-flow POC remains +- [x] Targeted POC(s) run + summary (OQ-TN-10) — UDP POC DONE + (2026-09-06, `poc-summary.md`); reverse-flow POC DONE + (2026-09-07, `reverse-poc-summary.md` — the `-R` shape validated + end-to-end over alkcall 0.6.0; findings F-1/F-2; W1 filed for + the upstream ledger). Remaining items (#3 unix/stdio, #4 helper + spike) are spec-scope: #3 folds into the real crate's tests + (pump agnosticism confirmed twice), #4 resolved by alkcall 0.6.0 + (ADR-050) - [x] Two-pump helper convergence input gathered (OQ-TN-06) — the POC gives both shapes (producer `pump_halves`, consumer `take_halves` + copy): same shape modulo channel side; the diff --git a/docs/research/poc-summary.md b/docs/research/poc-summary.md index 9e4365a..caf48d5 100644 --- a/docs/research/poc-summary.md +++ b/docs/research/poc-summary.md @@ -179,6 +179,9 @@ shape. Filed as a follow-up for the alkcall ledger. EOF is the channels-level `length=0` chunk on the `BiStream`. The two coexist without ambiguity — the consumer's `recv_datagram` returns `None` only on stream EOF, never on an empty datagram. + **Mandate strengthened 2026-09-07** (reverse POC F-2): without the + codec (raw pass-through), an empty datagram IS a zero-byte read — + colliding with EOF at the pump level. UDP must ride the codec. - **MTU discipline:** the codec rejects >65535 at frame time; the 1400-byte (max ethernet MTU payload) datagram round-trips through the bounded-buffer path. The E-04 64-parked-chunks bound was noted @@ -270,7 +273,12 @@ review and ADRs: JSON representation. The `OpenHandler` gains a `plan` parameter and the wrapper threads it. The POC's `HandleHandoff` side-channel and its same-resource race are dead: the establisher returns the dialed - handle directly (`Ok(Establishment::new(dialed))`). + handle directly (`Ok(Establishment::new(dialed))`). **Confirmed + under concurrency 2026-09-07** by the reverse-flow POC + (`reverse-poc-summary.md`): two concurrent opens of the same + resource, distinct channels, distinct dialed handles — the race + this handoff had is structurally gone. (F-1 there: plan payloads + must be `Send + Sync`.) 3. **Two-pump helper (→ R-03, bonus)** — `alkcall::channels::pump_bidi` pinned upstream (ADR-050), with `(u64, u64)` copy counts (no `io::Result` — errors are EOF-shaped per the ADR-078 contract). The diff --git a/docs/research/reverse-poc-summary.md b/docs/research/reverse-poc-summary.md new file mode 100644 index 0000000..9413d35 --- /dev/null +++ b/docs/research/reverse-poc-summary.md @@ -0,0 +1,237 @@ +--- +status: complete +last_updated: 2026-09-07 +--- + +# alktunnels: Reverse-Flow (`-R`) Tunnel POC Research Summary + +**Status:** Research complete — the `-R` shape (the hub opens tunnel +channels *toward* a connect-side worker that serves its own open op) +validated end-to-end over alkcall 0.6.0. 14 tests pass; clippy +`-D warnings` clean; fmt clean. The review 007 non-finding trace +("no upstream mechanism is missing") is confirmed by execution — +and the POC surfaced two real findings the forward POC could not +(one an upstream sharpness question, one a spec-mandate). +**Date:** 2026-09-07 +**Scope:** Validates the OQ-TN-10 #2 POC — the SSH `tcpip-forward` +template (register → per-accept channel-open → cancel) mapped onto +alkcall's both-sides serving semantics. Probes four wrinkles (W1..W4) +selected from the mechanism trace; findings F-1/F-2 below. + +--- + +## Executive summary + +A POC (`alktunnels-reverse-poc`, `/workspace/alktunnels-reverse-poc`) +validated the reverse flow end-to-end over alkcall 0.6.0: + +1. **The worker (connect side, serving side)** — dials the transport, + `ChannelClient::from_connection_with_serving` + + `ChannelOperations::register_on` (generic channel ops on the + serving registry, so the hub can `channel/close`) + post-hoc + `register_openable_with_establisher` (W2). Its establisher dials + the resource target and returns the handle via + `Establishment::new(plan)` — R-01's plan flow, no handoff map. + Its pump handler awaits `alkcall::channels::pump_bidi` inline + (R-02/R-03). This is ADR-022 §2 both-sides semantics exercised + for real: connect side AND serving side. +2. **The hub (accept side, reverse-flow initiator)** — + `ChannelsAdapter` + install hook that captures the hub's channel-0 + `CallConnection` and `ChannelManager`. Per local accept: call the + worker's open op (`call_with_payload` + `auth_token`), adopt the + worker-allocated channel ID, pump the accepted socket against the + adopted halves via `pump_bidi` spawned locally. +3. **The ADR-047 §5 orientation under test holds:** the reverse + channels are allocated by the worker (connect side — odd IDs) and + adopted by the hub (accept side). "Connection owner allocates" + exercised in the Pub-like orientation. +4. **14 tests:** TCP round trip, 1 MiB backpressure, half-close + semantics, concurrent same-resource channels (the R-01 race-killer + proven: two opens of one resource, distinct channels, distinct + dialed handles), UDP round trip, TCP+UDP concurrent, typed + establishment errors (`unknown_resource`, `dial_failed`, + `FORBIDDEN` ×2), out-of-band `channel/close`, worker outbound + calls resolving while serving, late-registration visibility, and + two finding-pinning tests (F-1 compiler-pinned, F-2 below). + +## Wrinkle probes (W1..W4) — results + +- **W1 identity on the serving path — CONFIRMED, upstream question.** + `from_connection_with_serving` builds channel 0 internally; its + `Connection::identity()` is empty and `ServingConfig` has no auth + capture point. The serving dispatcher resolves the caller identity + ONLY from the payload `auth_token` → + `ServingConfig.identity_provider` (verified: scoped token → open + proceeds; absent/foreign token → `FORBIDDEN`). The establisher and + pump handler receive the connection-establishment-time `AuthContext` + closed over at `register_openable` — NOT the per-call opener. + Consequences: (a) a scope-gated reverse-flow open op is satisfiable + today only via payload tokens (hub-side: attach + `auth_token` to `call_with_payload`); (b) transport-level + authentication (mTLS/QUIC peer identity) never reaches the + open-op ACL on the connect-side serving path. This is the same + asymmetry the accept side resolved with the install hook's + `channel0_conn.set_identity`. Candidate follow-ups: `ServingConfig` + gaining an identity capture, or documented token-only posture. + → alkcall review ledger (W1). +- **W2 post-hoc openable registration — works.** The worker registers + its openable AFTER `from_connection_with_serving` returns; the + dispatcher reads through the shared `Arc` per + dispatch. Proven by every open succeeding + a direct + `registration()` assertion. No finding — the assembly order + (construct client → register ops → serve) is viable as-is. +- **W3 adopter self-reaping — confirmed asymmetry, POC-local fix.** + The serving side's channel is wrapper-managed (teardown cascades on + pump completion or out-of-band close). The ADOPTING side has no + such machinery: `adopt_channel` installs routing state nothing + awaits. The hub must hold its pump handle and reap + (`teardown_channel`) itself — `ReverseTunnel::join_and_reap`/`close` + in the POC. Spec note: the real crate's consumer half needs a + session type with the same ownership (the adopted channel + pump + handle + reaper). Not an upstream gap (the hub's lifecycle is + assembly-layer by design, OQ-TN-04) — but it is the reason + `TunnelSession` in the spec must own teardown explicitly. +- **W4 EOF sentinel propagation — works both ways.** Half-close + semantics test: local `shutdown()` → EOF sentinel into the mux → + worker pump sees the zero-length read → shuts the target write + down; the target's echo still flows back before the reverse leg + completes. `pump_bidi`'s shutdown-on-completion contract holds + across the reverse path unchanged. + +## Findings + +### F-1 — `ChannelPlan`'s `Send + Sync` bound constrains plan payloads (compiler-pinned) + +`ChannelPlan = Arc` (ADR-049 amendment 2). +A dialed-socket handle whose halves are +`Box` does NOT satisfy the bound — +`Arc` requires `T: Send + Sync`, and `dyn AsyncRead` is not +`Sync` by default. The POC widened the halves to `+ Sync` (socket +halves are `Sync`, so dial-shaped payloads are unaffected). +Implications for Phase 1: tunnel plan payloads (target handles) must +be `Sync`, which is fine for socket-backed substrates but would +exclude non-Sync handles (e.g. process pipes as boxed trait objects) +without a wrapper. The spec should pin "plan payloads are +`Send + Sync`" as a documented constraint of the establisher API. +No upstream change needed — the bound is correct (the plan crosses +the wrapper task boundary). + +### F-2 — empty UDP datagrams collide with the EOF sentinel in raw pass-through (spec-mandate) + +The raw pass-through pump shape (`tokio::io::copy` between the UdpHalf +adapter and the channel) cannot carry an empty datagram: `poll_recv` +returning zero bytes is `Ok(0)` from the adapter — indistinguishable +from EOF for `copy`, which treats it as end-of-stream and shuts the +pump down. An empty datagram and the zero-length EOF sentinel are the +SAME wire shape at the pump level. Pinned by test +(`reverse_udp_empty_datagram_finding` asserts the non-round-trip so +the conclusion is executable, not anecdotal). +Consequence: the forward POC's codec decision — `[len: u16 BE]` +framing for UDP — is now validated from a SECOND angle: not only does +it preserve boundaries, it is MANDATORY for correctness (an empty +datagram is 2 bytes under the codec, unambiguous with EOF). The +Phase 1 spec must state: UDP rides the length-framed codec, never raw +pass-through. Stream substrates (TCP, unix) keep raw pass-through — +a zero-length read there is genuinely EOF. + +## The `-R` registration template (SSH tcpip-forward analogue) + +SSH: global request `tcpip-forward` → per-accept `forwarded-tcpip` +channel opens toward the forwarder → `cancel-tcpip-forward`. + +Mapped onto the validated surface (OQ-TN-03's role-follows-resource +model — no protocol-level direction): + +| SSH | alktunnels reverse shape (validated) | +| --- | --- | +| `tcpip-forward` registration | hub binds the listener locally (OQ-TN-04: binding is assembly-layer; no advertisement op needed for the hub's OWN listeners) | +| server-side accept loop | hub accept loop (assembly layer) | +| per-accept `forwarded-tcpip` open | hub calls the worker's served open op per accept (`call_with_payload`) + `adopt_channel` | +| target dial | worker's establisher dials the resource (establishment phase — typed errors) | +| `cancel-tcpip-forward` | hub closes: pump teardown + `teardown_channel` (and/or `channel/close` out-of-band on the worker) | +| originator pair (informational) | not needed — ACL rides the channels open-op machinery | + +Key validation: **no advertisement surface is needed in the base +crate.** The "please listen on your side" step of SSH's template +dissolves when the listener is the hub's own local resource — the hub +does not ask the worker to listen; the worker serves open ops, and +the hub opens channels whenever its own accepts fire. An advertisement +op only becomes necessary if the LISTENER lives on the far side +("worker, expose port 8080 on your end") — which is the worker +exposing a produced resource = the forward shape with the listener as +the resource (producer-side listen is an establisher variant, not a +new mechanism). This closes the last open thread of OQ-TN-03's +original question set (the `-R` advertisement/lifecycle surface). + +## What was built + +``` +alktunnels-reverse-poc/ + Cargo.toml — alkcall 0.6.0; tokio wasm-clean subset + net (POC-local) + src/ + lib.rs — module docs; wrinkle overview + producer.rs — worker half: tunnel_open_spec, tunnel_establisher + (Establishment::new(plan) — R-01), make_tunnel_ + pump_handler (pump_bidi inline — R-02/03), + register_tunnel_openable (post-hoc, W2) + producer/ + params.rs — TunnelParams {resource, substrate} + the UdpHalf + adapter (Send + Sync — F-1) + consumer.rs — hub half: open_reverse_channel (call_with_payload + + auth_token — W1), ReverseTunnel::open_and_pump + (adopt + pump_bidi), join_and_reap/close (W3), + bi_stream_from_halves + harness.rs — wire(): worker = from_connection_with_serving + (TokenIdProvider — W1) + ChannelOperations:: + register_on + post-hoc openable; hub = adapter + + capturing install hook (manager + CallConnection) + tests/ + tunnel_poc.rs — 14 tests (see Executive summary) +``` + +## What the POC does NOT validate + +1. **A real transport** — `tokio::io::duplex` stands in for + TCP/TLS/QUIC (unchanged from the forward POC scope). +2. **The advertisement op for far-side listeners** — resolved as + "not needed" (see the template table); if a deploy mode ever wants + "worker, expose a port on YOUR side," that is a producer-side + listen establisher over the same open op — spec-shaped, not POC- + shaped. +3. **Unix socket + stdio substrates** — OQ-TN-10 #3 (cheap; the pump + is substrate-agnostic and both POCs now confirm it). +4. **W1 remediation** — only the token path is exercised; whether + alkcall should grow a serving-side identity capture is an upstream + question (W1), not resolved here. +5. **Multi-hop (hub relay traversal)** — single hop; relay forwarding + (alkcall ADR-042) is transparent per leg. +6. **Wasm** — POC-local `tokio/net`; the real crate's protocol-only + core stays wasm-clean (unchanged conclusion). + +## Verification + +``` +14 tests pass; clippy --all-targets -D warnings clean; fmt clean. +Repeat-run stable (3× clean). +``` + +## References + +- alkcall 0.6.0 — review 007 remediation as consumed here: ADR-049 + amendment 2 (`Establishment::new(plan)`, R-01), ADR-050 + (`channels::pump_bidi`, R-03), R-02 JoinHandle contract. +- alkcall review 007 §Part B — the reverse-flow non-finding trace + (`from_connection_with_serving` + `register_on` + serving-side + allocation) — confirmed by execution. +- alkcall ADR-022 §2 — both-sides serving semantics (the worker's + shape). ADR-047 §5 — connection-owner allocation (the odd-ID + orientation). ADR-037/ADR-042 — channel ops, relay. +- alktunnels `docs/research/phase-0-findings.md` — OQ-TN-03 (direction + semantics — the advertisement thread this POC closes), OQ-TN-04 + (no forced binding — the hub binds, the worker never does), + OQ-TN-09 (half-open residual — validated W4), OQ-TN-10 #2 (this + POC). +- alktunnels `docs/research/ssh-socks5-survey.md` §RFC 4254 §7.1 — + the `tcpip-forward` template mapped in the table above. +- forward POC (`poc-summary.md`) — the codec decision F-2 strengthens; + the UdpHalf adapter reused; the plan flow replaces `HandleHandoff`. \ No newline at end of file