alktunnels/docs/research/tunnels-graduation.md

status, last_updated

status	last_updated
decided	2026-09-16 (§2 → ADR-007; §3/§3.1 → ADR-008 — the leans are pinned, see the status note)

alksocks research — graduating proxy surface into alktunnels

Status note (2026-09-16). Phase 1 decided both asks; this doc is the research record, the ADRs are now normative:

• §2 (the direct op) → ADR-007 — option (a) confirmed, with one refinement: the target encoding is a self-describing JSON object keyed by the sibling substrate field ({host, port} / {path}), not the ATYP byte string — the registry's input_schema validation (alkcall ADR-047) needs schema-describable params, and the unix path form falls out for free. Op id channels/tunnel/direct, scope tunnel:direct, dial-only.
• §3/§3.1 (listen metadata) → ADR-008 — the §3.1 forwarded-tcpip alternative ADOPTED over the per-ask carriers: a bind-first listen establisher (reply carries the real bound address — ask #1; also dissolves the POC's deadlock and 10s-bound findings) + the forwarded op channels/tunnel/forwarded with {listen, substrate, peer} params (ask #2; ADR-008 Amendment 1, 2026-09-18 — the correlation key is a producer-minted token, the original channel-ID key was unimplementable against the landed handler surface). The TargetHandle.peer option (§3 ask #2, option i) is superseded.
• A discovery this research missed: alkcall ADR-047 pins discovery's ALPN derivation to exactly …/sub/…/pub op-id shapes (derive_alpn_from_op_name) — flavor-form op ids (both new ops) need an upstream alkcall amendment before the first consumer. Filed: alkcall/docs/reviews/008-tunnels-graduation-upstream-review.md (both asks landed in alkcall 0.8.0, plus the in-tree ChannelRelay/HubLegTemplate — alkcall ADR-051).

The upstream ask ledger (§5) is now: items 1–2 pinned by ADR-007/008 (implementation pending the two upstream alkcall asks), item 3 unchanged (fast-socks5, courtesy). The Phase 7 "open items" are all resolved except OQ-SK-08's alksocks-side decision (item 3 of the ask ledger), which is unchanged by this.

The downstream motivation (2026-09-15 discussion): alksocks' three commands (CONNECT / BIND / UDP ASSOCIATE) need only a few alktunnels additions, and those additions are generally useful — they are the ssh -L/-R generalized shape that any dynamic-target consumer (SOCKS5 included) composes on top of. This doc records the proposed graduation: what moves to alktunnels, what stays in alksocks, the design questions each piece raises, and how the whole thing feeds the vpn-like endgame (§The vpn-like endgame) and the planned alknodes assembly layer.

Prior reading (in the alksocks repo, /workspace/@alkdev/alksocks/docs/research/): phase-0.md (the OQ ledger + the POC register), poc-bind-findings.md (the BIND ask ledger), poc-udp-associate-findings.md (the composed-egress shape + open-cost measurement), poc-connect-wasm-findings.md (the dial-at-command-read layering).

1. The precise gap (what "graduate the proxy" means mechanically)

Mapping the SOCKS5 command surface onto alktunnels 0.1.0:

SOCKS5 piece	alktunnels today	Gap
CONNECT (tcp proxy, egress)	dial establisher + pump_bidi + local::dial_tcp	none for local egress; no dynamic-target open for composed egress — params are {resource, substrate} (ADR-001)
UDP ASSOCIATE (udp proxy)	connected-udp substrate + per-target tunnel table	none — the alksocks phase-0 note "no upstream ask exists here" holds; the composed form is per-target udp opens (POC #2 measured ~0.8 ms/destination)
BIND (reverse tcp)	listen_establisher + AcceptQueue + bind_tcp	the two ledger asks: bound-addr discovery (reply#1's BND.ADDR) and peer metadata (reply#2's) — alksocks poc-bind-findings.md §Upstream-ask ledger

So the graduation reduces to:

1. One wire addition: a direct/dynamic open form — "dial an arbitrary target," not "dial a named resource" (§2).
2. The two BIND metadata asks — carriers to be decided (§3).

The pumping, teardown, codec, and session machinery are already done and POC-validated; the proxy producers on the alksocks side are thin adapters over the three callback seams (DialFn, EgressFactory, BindAcceptFactory), which the POCs validated in both local and composed variants.

2. The direct open form — the real design decision

The constraint behind it

alktunnels ADR-001 deliberately pinned params as "resource, not an address": the producer owns the backing; rich addressing rides the -D/dynamic composition path (inside the payload), never the wire. That decision was correct for its use case and the no-forced-bind posture (AGENTS.md #9) stands: nothing here forces any side to bind a port. The use case that justifies an additive revisit is the one the POC ledger itself names: a consumer asking the producer to dial an arbitrary host:port — ssh's direct-tcpip shape, which ssh carries as its own channel type precisely because it is a different capability from opening a named resource. SOCKS5 CONNECT is the same capability: a dynamic target chosen at session time.

The capability is dangerous in exactly the way ADR-001's "resource, not address" framing protects against — arbitrary egress from the producer's network. The mitigation is not refusing the capability (a SOCKS5 service IS the capability) but separating it at the ACL: "dial postgres-primary" and "dial anything from my network" must not be the same grant.

Options

(a) Second op on the same ALPN — channels/tunnel/direct with params {substrate, target} and its own scope (tunnel:direct). Clean in kind: this IS a different operation — dial what I name, vs. dial what the producer registered. The ACL separation is structural (scopes attach to op specs, not params); the POC #5 scope gate (socks5:bind) proved the pattern matters. Old producers reject the unknown op loudly (registry lookup fails — the SSH posture).

(b) Optional target field in the existing params + a target-implies-scope establisher check. ADR-001's consequences bless additive optional fields, but it couples the ACL to a field (scopes attach to the op), muddies the resource invariant, and forces the establisher to branch on the params shape. Weaker.

(c) Resource-name convention (resource = "tcp:host:port"). No wire change, but the resource id becomes ACL-blind (the registry grants by name; "any name of this shape" is not a scope), registry bookkeeping gets weird, and discovery listings leak addresses. Reject.

Lean: (a). One ALPN, two ops, two scopes — the same shape alktty's scope-gating and the POC #5 command gates took. Wire-stable questions (op id string, scope string, params field names, target address encoding) are pinned by an ADR before the first consumer — timing is right because alksocks is the first real consumer and nothing is published yet.

The target-encoding sub-question

The direct op's target must carry what DialFn today gets from the registry's opaque backing string: {host, port} for tcp, {host, port} (or resolved-destination semantics) for udp, {path} for unix. The RFC-1928 ATYP encoding (IP v4/v6/domain + port — TargetAddr's to_be_bytes) is the natural cross-substrate carrier: self-contained, no DNS dependency on the consumer side, and it is the encoding SOCKS5 itself uses — the alksocks adapter maps it verbatim. Domain-form targets stay unresolved on the wire (producer-side resolution, the ssh -D semantic; POC #1/#2 both validated egress-side resolution).

What does NOT move to alktunnels

The RFC 1928 conversation, reply codes, the command scope gates, the front doors, the {addr, data} datagram stage — all alksocks. The direct op is substrate-agnostic transport capability; it knows nothing about SOCKS5 (no fast-socks5 dependency, no reply-code semantics upstream). The layering assertion holds: alktunnels stays SOCKS-ignorant; alksocks composes it.

3. The BIND metadata asks — carriers, and one alternative

alksocks' poc-bind-findings.md pinned two asks. The POC's stand-ins (factory carries the expected bind_addr; sentinel 0.0.0.0:0 as reply#2's peer) are honest interims, so the design question is how to retire them.

The load-bearing constraint

The open-op reply is the only pre-data-plane wire moment, and it happens after the establisher. SOCKS5 BIND is address-first (reply#1 must carry BND.ADDR before the app server dials); alktunnels' listen establisher is accept-first (its establishment phase pops an already-accepted connection — the open resolves because the app server dialed). The POC resolved the sequencing in shape (reply#1 first; the tunnel open rides the accept future) — the asks are about fidelity, not feasibility.

The two asks, restated as carriers

Ask #1 — bound-addr discovery without consuming an accept. Options:

• Inspect-only open / listen/addr query op — the plan carries local_addr, or a small query op on the listen resource. Wire addition (a query op is a new op, but Query-typed — cheap, no data plane).
• Out-of-band discovery — the OperationSpec.description round-trips through discovery (OQ-TN-08); the producer's registry description carries the advertised address. Zero new wire; the cost is staleness (the description is install-time, the address may be OS-chosen) and per-session dynamic listeners have no description.
• Same-process assembly wiring — the POC's stand-in formalized: the alksocks producer's accept factory holds the far side's local_addr directly when it constructed the listen resource itself. Works only when the producer owns both halves in-process — which is the common hub case (alknodes re-producers), but not the general composition case.

Ask #2 — peer metadata on accepted handles. Impossible to carry in the open-op reply (post-reply#2 does not exist as wire); the option space is (i) TargetHandle gaining an optional peer: Option<SocketAddr>-shaped metadata field (additive, no wire change — the metadata rides the establishment phase, which is where the accept happens), or (ii) accept-as-open: the accepted connection opens a reverse channel whose open-op params carry the peer (§3.1).

3.1 The alternative worth weighing: the forwarded-tcpip shape

ssh's -R far-side listener opens forwarded-tcpip channels back toward the client, with the listen target + peer address riding the open params. alktunnels has the whole reverse path already (open_reverse_channel, TunnelSession::adopt, producer-allocated channel IDs — consumer.rs). The shape:

1. The alksocks producer, on BIND, opens the listen-tunnel (or binds locally) and advertises reply#1 from whatever address source the carrier question resolves.
2. When the far-side listener accepts, the tunnel producer opens a reverse channel toward the alksocks producer with {listen_target, peer} in params — peer fidelity for free.
3. The alksocks producer correlates the reverse open to the waiting BIND session (a session-id ↔ quiet-wait-task registry), writes reply#2 with the real peer, and pumps.

Costs: a correlation registry on the SOCKS5 side; a direction flip in the composed flow (the accept-initiated open is toward the alksocks producer); the quiet wait must accept opens, not await a popped handle. Benefits: both asks' fidelity with zero special carriers — the metadata rides the open op, which is the wire's native "this connection is for X" moment. This is the wire-native answer and should be compared against the two-ask ledger rather than assuming the ledger's shapes.

Noted asymmetry: the local accept variant needs neither carrier (the listener and the accept are in-process; reply#1/#2 come from the real sockets, POC-validated). Both asks exist only for the composed variant. That suggests scoping the asks to the composed variant and letting the local variant stay as-is.

4. The callback→provider mapping (what alksocks ships)

The three seams stay callbacks; the graduation gives each a composed provider next to the local-socket one:

alksocks seam	local provider (local feature)	composed provider (alktunnels)
DialFn (CONNECT)	dial_tcp/dial_unix (local)	open channels/tunnel/direct (or the direct-op form chosen) toward the target
EgressFactory (ASSOCIATE)	one UdpSocket per association (POC #2's LocalEgress)	per-destination udp tunnel table (POC #2's ComposedEgress — the direct op dissolves its resource_of stand-in)
BindAcceptFactory (BIND)	real TcpListener per session (POC #5's local variant)	listen-tunnel session (POC #5's composed variant — the metadata carrier decides reply#1/#2 fidelity)

All six paths are individually POC-validated; nothing here is hypothesis-level except the metadata carriers (§3).

Hub-relay transparency. The terminate-and-re-produce hub (alkcall ADR-042, alktunnels' refinement) forwards tunnel data byte-for-byte. One structural note for the ADR: a hub that re-produces the direct op applies its OWN target policy at the re-produce hop (the hub's ACL, not the original opener's, governs the final dial) — correct-by-default (the hub is the network boundary) but worth pinning as the security model sentence.

5. The ask ledger, reconciled

The one-version alktunnels bump, post-graduation:

1. The direct op (§2) — the wire addition itself: op spec, scope constant, params schema, establisher shape (dial only — no listen/direct-BIND form), DialFn reuse.
2. Listen metadata (§3) — bound-addr discovery + peer metadata, carriers per the ADR; OR the forwarded-tcpip shape as the combined alternative (§3.1). The POC ledger's two items map onto this item either way.
3. (alksocks courtesy, fast-socks5) — unchanged from POC #5's ledger: publish new_reply; the net feature gate; run_udp_proxy_custom's unconditional bind.

Item 3 is independent; items 1–2 are the single alktunnels version bump alksocks composes against.

6. The vpn-like layer — where this ends up

The graduation is the mechanical half of the vpn-like story; the addressing half rides on top:

• Virtual addressing (OQ-SK-05's addendum, now concrete). With a direct op, "map node/service → private IP" becomes a dial-callback translation: intercept connect(10.x.y.z:p), match the subnet against a per-identity address table, open the direct op toward the owning node's resource (or the named resource the address maps to). No wire change — DST.ADDR/BND.ADDR are arbitrary RFC fields; the intercept point is the SOCKS5 front door (tun2proxy routes the host's traffic there, --proxy socks5://... natively). The POC #2 resource_of seam is the same translation at the datagram stage.
• The signing half is not a tunnels concern. Attesting "node N serves 10.1.2.3:5432 in scope S" is a directory/identity-layer pattern (alkcall identities + a signed address→service table); alktunnels/alksocks consume it as dial-callback policy. Worth a prior-art check on tun2proxy's virtual-DNS/virtual-network support (alksocks phase-0 flagged this) when that layer is specced.
• alknodes (the planned assembly layer). The hub/spoke shape the user describes — spokes expose resources (postgres, redis, containers), the hub re-produces them to privileged spokes, spokes can be hubs themselves (russian-doll: an internal private network with selective exposure) — is the terminate-and-re-produce story alktunnels' OQ-TN-12 deferred until a concrete consumer exists. alknodes is that consumer. What it needs from this graduation: the direct op (dynamic egress through the overlay), the listen surface (accept-side resources), and the metadata carriers (BIND fidelity through one or more hops). What it adds on its own layer: resource directory, identity→subnet mapping, hub ACL policy — all assembly, no new wire.

7. Open items for Phase 1 / ADRs

• The direct op's wire shape (op id, scope string, params encoding) — one-way door; ADR before the first consumer.
• The listen-metadata carriers (or the forwarded-tcpip alternative) — same gate.
• Whether BIND ships in alksocks' base crate + default accept variant (OQ-SK-08's remaining decision) — unchanged by this doc, but the carrier choice feeds it.
• The scope-separation posture: tunnel:direct ≠ tunnel:open ≠ ingress scopes; pin the convention sentence in the ADR.
• The hub re-produce security-model sentence (§4).