alktunnels/docs/architecture/producer.md

status, last_updated

status	last_updated
draft	2026-11-03 (the forwarded op's serving-side establishment phase + the correlation-miss vocabulary, ADR-008 Amendment 2)

alktunnels: Producer Half

The producer half: the channels/tunnel/sub open op, its spec, its establishers (dial + listen shapes), its pump handler, and its registration, plus the sibling ops on the same ALPN: the direct op (channels/tunnel/direct, dynamic-target egress) and the forwarded op (channels/tunnel/forwarded, accept-as-open). The producer is whoever can reach the tunnel target; connection direction is irrelevant (OQ-TN-03's hub model). The reverse POC validated the producer running on the connect side of its own connection (from_connection_with_serving).

Design decisions: ADR-001 (params), ADR-002 (ALPN), ADR-003 (codec), ADR-004 (no backend trait), ADR-007 (the direct op), ADR-008 (listen metadata, bind-first, the forwarded op). This document is the normative WHAT.

The Open-Op Spec

operation id:   channels/tunnel/sub
op type:        Sub          (reply = one { channel_id } envelope; data plane on the BiStream)
ALPN marker:    alk/tunnel   (ChannelOpenSpec)
visibility:     External
input schema:   { resource: string (required), substrate: enum [tcp, udp, unix] (required) }
output schema:  { channel_id: integer > 0 }
access control: required_scopes: ["tunnel:open"]
description:    SHOULD carry a human-readable hint (round-trips through discovery, OQ-TN-08)

Registration is via ChannelCore::register_openable_with_establisher (alkcall ADR-049 §5): the registry schema-validates input, runs the ACL, allocates the channel (connection-owner rule, ADR-047 §5), runs the establisher as the awaited bounded establishment phase, then spawns the pump handler. The generic channel ops (channel/close, channel/control, channel/resources/subscribe) register separately via ChannelOperations::register_on; the producer (or its assembly layer) MUST register them on the serving registry so peers can manage channels out-of-band (the recipe alkcall review 007 §Part B verified; see alkcall/docs/reviews/ 007-establishment-follow-ups-review.md; R-01 = the Establishment plan payload, R-02 = the OpenHandler lifetime contract).

Registration timing: the openable may register AFTER from_connection_with_serving returns (the connect-side serving posture; the reverse POC's W2): the dispatcher reads through the shared Arc<OperationRegistry> per dispatch. Late registration is a supported assembly shape, not a race.

The Direct Op (channels/tunnel/direct)

Dynamic-target egress (ADR-007): the same ALPN, a second Sub-typed open op whose establisher dials a per-call target instead of resolving a resource. One ALPN, two ops, two scopes: dial-what-the-producer- registered (tunnel:open) and dial-anything (tunnel:direct) are different capabilities.

operation id:   channels/tunnel/direct
op type:        Sub          (same reply/wrapper shape as …/sub)
ALPN marker:    alk/tunnel   (ChannelOpenSpec)
visibility:     External
input schema:   { substrate: enum [tcp, udp, unix] (required),
                  target: object, per-substrate sub-schema (required) }
output schema:  { channel_id: integer > 0 }
access control: required_scopes: ["tunnel:direct"]
description:    SHOULD carry a human-readable hint (dynamic-target egress)

• No registry lookup: the establisher parses target against substrate (malformed → invalid_input-class; unknown_resource can never fire) and injects the parsed TunnelTarget into the same assembly-provided DialFn the base op's backings resolve to.
• Dial-only: no listen/direct-BIND form; the ingress capability is the listen + forwarded surface (ADR-008).
• Registration: register_tunnel_direct_openable (same wrapper, same plan-flow; the spec builder is public for assembly variants).

The Establisher

The establisher is the open op's awaited, bounded establishment phase (alkcall ADR-049 §1/§2): validate params semantically, consult ownership if wired, dial/accept the substrate, and return Establishment::new(plan); the dialed handle rides the typed-opaque plan (ChannelPlan = Arc<dyn Any + Send + Sync>; the + Sync bound constrains plan payloads; socket-backed handles carry it naturally, reverse POC F-1, documented upstream). On failure: EstablishmentError (the typed reason codes) → the wrapper tears the channel down and replies channel:open_failed.

Three shapes (all return the same plan-flow; ADR-004):

1. Dial establisher (the -L/forward shape)

Registry lookup of (resource, substrate) → backing → dial → halves. The dial function is substrate code injected at assembly (ADR-004: the establisher closure closes over it); the protocol crate never sees a socket type. Failure mapping:

Producer condition	Typed reason
resource not in the registry (or wrong substrate)	unknown_resource
dial refused / timed out / unreachable	dial_failed
bind/connect resource exhaustion (fds, ports, slots)	resource_shortage
establisher-internal failure	handler_error
the bound exceeded	timeout (wrapper-level)

The direct establisher is this shape minus the lookup row: target malformed for the substrate is the schema/invalid_input-class failure (ADR-007 §4's table).

2. Listen establisher (the producer-side listener variant)

For a producer whose resource is a LISTENER (SSH -R far-side listener; a hub exposing its own port to a third party): the same open op, the same params; the establisher pops the next accepted connection from an assembly-owned listener queue and returns it as the plan payload. The listener lifecycle (bind, accept loop, cancel) is assembly code; the protocol never binds (OQ-TN-04). Error mapping: an empty queue during establishment is resource_shortage; a closed listener is dial_failed-class. No new wire surface: the listen variant is plan-flow with a different halves source.

The accept-first shape stays (ADR-008 §5): for producers whose resource backing IS an accept queue (an externally-fed or pre-bound listener shared across opens), this remains the correct shape: the POC #5 composed-BIND shape. Its limits (no bound in the reply: the accept hasn't happened at establishment; the POC's establishment-bound-vs-wait problem) are inherent to accepting-first.

3. Bind-first listen establisher (the -R/ingress shape, ADR-008)

For a producer that OWNS the bind decision (the common hub case; alksocks' composed BIND): the establisher binds the listener (via the assembly-provided listen function; the protocol never binds, OQ-TN-04) and returns the listener handle as the plan payload. Establishment is fast (bounded by the bind, not by any app-server dial):

• The establishment phase ends at bind time, so the open-op reply can carry the REAL OS-chosen address: the establisher contributes "bound": {host, port} (or {path}) to the reply: the establisher → reply projection (ADR-049 amendment 3, Establishment::with_reply_field; the channel_id key is wrapper-reserved). Landed in alkcall 0.8.0 (review 008 U-2); consumers read it via ChannelClient::open_channel_with_reply.
• The establisher mints the correlation token (ADR-008 Amendment 1): a fresh u32 per open, contributed as the "listen" reply field (the same projection) and carried in the plan; the accept dispatcher reads it from the plan and stamps it into each forwarded open's listen param. The plan carries {listener, token, substrate} (a bind-first plan type; the substrate is stamped at establisher time, and the dispatcher reads it from the plan, never re-parses wire input); the token is relay-transparent by construction (reply fields and params both pass hubs untouched per alkcall ADR-051; ADR-008 Amendment 1 has the why-superseded of the channel-ID key).
• The pump handler for this shape is an accept dispatcher, not pump_bidi: it accepts the channel's BiStream (the R-02 contract), then runs the assembly-provided accept loop; each accepted connection produces a forwarded-channel open toward the listen opener (§The Forwarded Op below); the accepted halves pump against the adopted channel. The listen channel's own BiStream stays idle; its EOF (the opener dropping the listener session) cancels the loop and tears the listener down.
• Error mapping: unbindable/held port → dial_failed-class (the bind IS the dial here); bind resource exhaustion → resource_shortage; the rest of the table unchanged.
• The address-first-vs-accept-first BIND deadlock (the alksocks POC's one real discovery) DISSOLVES in this shape: the open completes at bind time, reply#1-class metadata (the bound field) precedes any app-server dial, and the accept wait lives in the pump phase: teardown-governed, not establishment-bounded (the 10s bound problem dissolves with it).

Identity: the establisher receives the per-call opener identity (the dispatch-resolved identity overlaid onto the install-time context, alkcall 0.7.0 CF-006). On hub-forwarded opens it sees the end client, not the hub. auth carries the opener's identity plus the transport-truthful fields (alpn, remote_addr, tls_client_fingerprint); these are never rewritten. Consult it for ownership-scoped resources if the assembly layer wants OwnershipProvider checks here (OQ-TN-08's optional half).

Concurrency: each open's establisher result flows to ITS pump handler; concurrent opens of the same resource are independent (the R-01 plan flow per alkcall review 007 R-01, the Establishment::new(plan) payload; the forward POC's resource-keyed handoff race is structurally gone, confirmed under concurrency by the reverse POC).

The Pump Handler

The OpenHandler (alkcall ADR-049/050 shape: Fn(Value, Option<ChannelPlan>, Connection, AuthContext) -> JoinHandle<()>):

1. Downcast the plan to the concrete handle type (Arc::downcast::<TargetHandle>, so establisher and handler agree on the type; alkcall never inspects it).
2. conn.accept_bi(): the channel's yield-once BiStream.
3. pump_bidi(bidi, handle.read, handle.write): awaited inline inside the handler's task. The returned JoinHandle MUST track the data-plane lifetime (R-02): the wrapper awaits it and its completion triggers channel teardown. Early return = teardown-at-birth: both pumps see instant EOF with zero bytes; the "spawn-and-forget" shape is the one bug class the R-02 telemetry (yield-once flag + birth-teardown debug hint) diagnoses at runtime. Hand-rolling the two-pump shape is out (ADR-050).
4. UDP: the plan payload is the FRAMED adapter (ADR-003), so the pump is unchanged; the codec lives at the substrate/pump boundary.
5. Bind-first listen handler (the accept dispatcher, ADR-008 §1): the plan payload is the listener; after accept_bi(), the handler runs the accept loop (assembly-provided) concurrently with holding the channel's BiStream; each accepted connection produces a forwarded-channel open toward the listen opener. The listen channel's own stream stays idle (no pumps on it); its EOF cancels the loop (the listener tears down with the session). Per-accept data planes are still pump_bidi (convention 8, per-channel, never a hand-rolled loop over one stream).

The handler is substrate-agnostic by construction: it cannot know whether the halves came from TCP, UDP, a unix socket, or an in-process pipe, and must not (AGENTS.md convention 7).

The Forwarded Op (opener side, ADR-008 §3 as amended by Amendment 2)

Accept-as-open: when the bind-first listener accepts, the PRODUCER opens a channel back toward the listen opener. The opener SERVES the op (channels/tunnel/forwarded, scope tunnel:forwarded, ingress; Amendment 2's two serving shapes: the recommended correlation-gate establisher variant, where a stale/missing token fails the open as a typed channel:open_failed/unknown_resource before the channel is born, or the base establisher-less shape, where the data plane starts immediately and the assembly's post-open decline owns the miss):

1. The producer's per-accept open carries {listen, substrate, peer}: the correlation token from the plan (the value the establisher minted; the opener matches it against the listen field of its open listener sessions per ADR-008 Amendment 1), the substrate (also from the plan, stamped at establisher time from the one successful params parse; the dispatcher never re-parses wire input), the accepted connection's remote address (reply#2-class fidelity's carrier).
2. The opener's registry allocates the channel_id; the producer adopts it (TunnelSession::adopt) and pumps the accepted halves against it (pump_against).
3. The opener-side handler correlates the open (via listen) to the waiting session (assembly-side bookkeeping; the protocol provides the op and params, not the wait state machine), accepts the BiStream, and pumps against the local session's halves. Under the establisher variant (Amendment 2) that correlation lookup rides the gate at establisher time instead; the handler-side correlation remains only for the narrow residual race (a token vanishing between gate pass and handler run; the miss class is named correlation-miss at debug, assembly-side).
4. Identity: the forwarded op's opener-side gate sees the PRODUCER's per-call identity (CF-005 precedence; the producer may carry an auth_token when it is itself forwarding); the peer address in params is producer-asserted data, not transport truth; the ACL trusts the producer's grant, and the application treats peer as metadata.

Registration API (the spec surface)

pub fn register_tunnel_openable(
    core: &ChannelCore,                 // the manager + policy pair
    registry: &ResourceRegistry,        // assembly-owned: (resource, substrate) -> backing
    on_registry: &Arc<OperationRegistry>,  // the session's dispatch registry
                                        // (the per-session fork per alkcall ADR-047 §4
                                        // amendment, 2026-09-03, NOT the connection
                                        // overlay; Layer-2-registered open ops resolve
                                        // NOT_FOUND, review 004 F-01)
    auth: AuthContext,                  // the install-time context (per-call overlay is upstream's)
    dial: DialFn,                       // substrate halves function (ADR-004; `local` feature or assembly)
) -> Result<(), String>

pub fn register_tunnel_direct_openable(
    core: &ChannelCore,
    on_registry: &Arc<OperationRegistry>,
    auth: AuthContext,
    dial: DialFn,                       // the SAME DialFn the base op uses; receives the parsed TunnelTarget
) -> Result<(), String>

pub fn register_tunnel_listen_openable(   // accept-first (shape 2, unchanged)
    ...                                   // as today: listen_establisher + AcceptQueue
)

pub fn register_tunnel_bindfirst_openable( // bind-first (shape 3, ADR-008)
    core: &ChannelCore,
    on_registry: &Arc<OperationRegistry>,
    auth: AuthContext,
    bind: BindFn,                       // substrate bind function -> (listener, bound addr)
    accept: AcceptFn,                   // per-accept: open the forwarded channel, pump
) -> Result<(), String>

- `ResourceRegistry` (assembly construct, OQ-TN-11): maps
  `(resource, substrate)` → backing. The POCs' `HashMap` shape is the
  candidate; the collision domain is the producer's registry
  (per ADR-001, two producers on one connection may expose the same
  resource name independently).
- The scope constants `TUNNEL_OPEN_SCOPE = "tunnel:open"`,
  `TUNNEL_DIRECT_SCOPE = "tunnel:direct"`, and
  `TUNNEL_FORWARDED_SCOPE = "tunnel:forwarded"` (ADR-006 Amendment 1);
  the open-op spec builders are public so assembly layers can compose
  variants (e.g. extra ACL fields) without forking the wire shape.
- The pump handler factory is public for the same reason.

## Access control posture

Tunnels reach local networks; the open gate is the security boundary
(OQ-TN-08, ADR-006 + Amendment 1):

- The registry runs `AccessControl` before the establisher (the ACL
  rides `register_openable`; free via alkcall).
- `TUNNEL_OPEN_SCOPE` is the base gate; `TUNNEL_DIRECT_SCOPE` gates
  dynamic-target egress (never implied by `tunnel:open`);
  `TUNNEL_FORWARDED_SCOPE` gates ingress (receiving forwarded
  connections) on the opener side. Three scopes, three capabilities.
- The assembly layer MAY add resource-scoped checks via the
  establisher (ownership on the resource name); the protocol
  provides the seam, not the policy. For the direct op the seam is
  the parsed `TunnelTarget` (a deployment MAY enforce target
  allowlists in its establisher; policy, not protocol).
- No target allowlists in the PROTOCOL, no tunnel-specific policy
  machinery in v1, as the far side owns the resource (OQ-TN-08
  resolution); discovery is the ACL-filtered ops listing.
- **Hub re-produce policy (the security model sentence, ADR-006
  Amendment 1):** a hub re-producing the direct op applies ITS OWN
  target policy at the re-produce hop; the hub's ACL governs the
  final dial. The hub is the network boundary.

## Open Questions

- **OQ-TN-11**: resource naming collision domain + lifecycle
  (partially resolved; the per-producer registry is the leading
  candidate)
- **OQ-TN-14**: resolved (split: unix in, stdio out, with alktty's pipe
  mode owning process stdio; see open-questions.md)
- **OQ-TN-15/16**: resolved (ADR-007/008: the direct op, listen
  metadata, the forwarded op)

## References

- [wire.md](wire.md) (the ops' wire surface),
  [overview.md](overview.md) (module map)
- Forward POC `docs/research/poc-summary.md`; reverse POC
  `docs/research/reverse-poc-summary.md` (both producer shapes ran)
- `docs/research/tunnels-graduation.md` (the graduation research;
  §3.1 is the shape bind-first + the forwarded op adopt)
- alkcall ADR-047/049 (open ops + establishment), ADR-050 (`pump_bidi`),
  ledger CF-005/CF-006 (identity precedence + per-call opener)
- Upstream asks (review 008): `alkcall/docs/reviews/
  008-tunnels-graduation-upstream-review.md`; both landed in
  alkcall 0.8.0 (flavor-form discovery derivation, ADR-047
  amendment 3; the establisher → reply `bound` projection, ADR-049
  amendment 3), plus the in-tree `ChannelRelay`/`HubLegTemplate`
  (alkcall ADR-051)