alkcall/docs/architecture/decisions/049-channel-open-establishment-phase.md

ADR-049: Channel-Open Establishment Phase (OpenEstablisher)

Status

Accepted — implemented in alkcall 0.5.0 (Unit 1: E-01 + N-1; see the "Amendment (Unit 1 implementation, 2026-09-06)" at the bottom). Amends ADR-047 §3 — the open-op wrapper gains an awaited establishment phase ahead of the spawned pump handler; resolves review 006 E-01 and N-1. Amendment 3 (2026-09-16, review 008 U-2) adds the establisher reply projection — see "Amendment 3" below.

Context

ADR-047 §3 made openable ALPNs operations: the ALPN crate supplies an open handler, and the channels wrapper does the channel machinery (allocation, ledger, policy, spawn). The §3 decision text describes the handler's job as "validate params, consult ownership, prepare the backend, return a 'channel plan'" — an awaited preparation step the wrapper consults before replying. The implemented OpenHandler type (Arc<dyn Fn(Value, Connection, AuthContext) -> JoinHandle<()>) collapsed that preparation into a fire-and-forget spawn: the wrapper collects the JoinHandle, records it for teardown, and writes { "channel_id": <id> } to the wire the moment the handler task is spawned (run_open_wrapper, src/channels/operations.rs). The establishment phase the ADR described never became a thing the wrapper could consult.

The consequence (review 006 E-01, verified at tree 88e3f5e): the open op cannot fail after allocation. Any establishment failure inside the handler — params valid at the schema level but semantically rejected, backend lookup failure, a target dial refused for a direct-tcpip-shaped tunnel, a resource no longer available — is invisible to the open reply. The consumer observes: the call op succeeds with {channel_id}, the channel is adopted, and then the channel EOFs (the handler exits without writing; the mux pump writes the implicit-EOF chunk; MpscRecvStream::poll_read returns clean EOF for both the sentinel and sender-drop arms). A dial failure is byte-for-byte indistinguishable from a target that closed immediately after connecting — the two most different failure/success stories map to the same consumer-visible event.

Every established tunnel/forwarding protocol puts establishment failure in the open reply, not in the data stream:

• SSH (RFC 4254 §5.1): SSH_MSG_CHANNEL_OPEN_FAILURE is a first-class reply carrying a reason code (ADMINISTRATIVELY_PROHIBITED / CONNECT_FAILED / UNKNOWN_CHANNEL_TYPE / RESOURCE_SHORTAGE) plus a description string; the channel never exists on the opener's side afterward.
• SOCKS5 (RFC 1928 §6): the reply carries REP codes 0x01–0x08; error-then-close, never "success then in-stream error."
• udpgw (tun2proxy) is the counterexample: an opaque ERR bit with zero reason information — the vocabulary to avoid.

The per-crate workaround proves the gap is load-bearing: alktty's channels path answers establishment failures with a length-prefixed JSON error frame on the channel stream (send_negotiation_error, alktty/src/adapter.rs), disambiguated from data by a 0x00 first-byte peek. That is a per-crate reinvention of a protocol-level capability every ALPN crate will need: a structured, typed, establishment-failure reply to the open op. It also forces the phantom-opened channel to exist in the manager — allocation, ledger, and policy all fire for a channel that never carries data.

alktunnels (the next consumer, arbitrary TCP/UDP tunnels over channels) hits this on day one: its producer dials the tunnel target inside the open op, and dial failure is the common case, not the edge case (review 006 E-01; alktunnels OQ-TN-09).

This is the cheapest moment to fix upstream: three downstream crates (alktty, alkhttp, alktunnels-in-progress), alkcall 0.4.x, no published consumer depends on the phantom-open shape.

Decision

1. The open-op wrapper gains an awaited establishment phase

ChannelCore::register_openable accepts an optional establisher alongside the existing OpenHandler:

/// The establishment result. `Establishment` carries what the
/// pump phase needs (today: nothing — reserved for a channel plan).
/// `EstablishmentError` carries the reason code + message the
/// wrapper puts in the open reply's `details`.
pub type OpenEstablisher = Arc<
    dyn Fn(Value, Connection, AuthContext)
        -> BoxFuture<'static, Result<Establishment, EstablishmentError>>
        + Send
        + Sync,
>;

The establisher is awaited by the wrapper, bounded — before the reply is written, before the pump handler is spawned. The pump handler (the existing OpenHandler, unchanged) is spawned only on establishment success. The dial is the natural establisher step for tunnels; the pumps remain the spawned handler. This restores ADR-047 §3's original "channel plan" shape: the establisher is the awaited preparation, the wrapper consults its result, the pumps are the spawned protocol.

Why split-hook, not await-and-inspect (the two candidates review 006 proposed):

• The split preserves the OpenHandler type exactly. Await-and-inspect changes OpenHandler's return type (JoinHandle<()> → JoinHandle<OpenResult>), breaking all three consumers' handlers and the alkhttp OpenableAlpn ferry for no compensating gain.
• Establishment and pump are genuinely different lifecycles. The dial is synchronous with the open reply (SSH's semantics: the failure is the open's reply); the pumps outlive the reply. Coupling the establishment signal to the pump task's lifecycle (watching a JoinHandle for a first resolution) conflates them and makes the "established, continue" signal an out-of-band convention (a sentinel Result value, a oneshot the handler must remember to signal) — more protocol per crate, the thing this fix exists to remove.
• The split is backward compatible by construction: no establisher registered = an always-OK establisher. Existing registrations compile and behave unchanged.

2. The deadline bounds the establisher, not the pumps

The establisher await is bounded by the dispatch deadline when the OperationContext carries one (context.deadline), else by a crate constant (ESTABLISHMENT_TIMEOUT, 10s default; overridable per registration via a Duration argument on the establisher-taking register_openable variant). On deadline expiry the wrapper treats it as establishment failure with reason timeout.

The bound applies only to the establisher. The spawned pump handler's lifetime is governed by the existing teardown machinery (channel/close, connection drop, handler exit) — unchanged.

Head-of-line safety is already proven: the serving loop spawns Once invocations as independent tasks (Dispatcher::spawn_once_dispatch), so a slow establisher on one open op does not block other calls on channel 0.

3. Establishment failure: teardown + typed channel:open_failed

On establishment failure (error or deadline), the wrapper:

1. Tears down the just-allocated channel (teardown_channel — drops the demux sender, returns the not-yet- installed handler task handle if any),
2. Takes the opener-ledger entry and calls policy.on_close(opener) (the same un-increment path the allocation-failure arms already run — the ledger take is the atomic gate, ADR-047 §7),
3. Replies with a new typed error:

code:     "channel:open_failed"
message:  human-readable establishment failure description
retryable: false
details:  { "reason": <reason-code>, "message": <detail string> }

The reason-code vocabulary maps 1:1 onto what an establisher can actually produce (per the SSH four; the survey's finding):

reason	meaning
dial_failed	the backend/target could not be reached or refused
unknown_resource	the requested resource does not exist
resource_shortage	the backend is out of capacity (ports, fds, slots)
handler_error	establisher-internal failure not covered above
timeout	establishment exceeded the deadline

Policy denial stays channel:too_many_channels (pre-allocation, unchanged); ACL denial stays FORBIDDEN (registry gate, unchanged). The new code is an additive wire addition (new error-code string + optional details shape); no existing consumer breaks. ALPN crates' open-op specs gain matching ErrorDefinition entries per ADR-016 so services/schema discloses the failure contract.

The SSH "channel never exists opener-side" property is the contract: the consumer's open resolves Err and no channel_id was ever returned. (The allocation still happened accept-side momentarily — that is invisible to the consumer and is what the teardown in step 1 cleans up.)

4. ChannelClient::open_channel stops erasing the error (review 006 N-1)

ChannelClient::open_channel currently flattens the CallError into a String (format!("open op failed: {e:?}")), which would make the typed reason invisible to consumers — the E-01 fix would be unreachable end-to-end through the primary client path. It changes to return a typed error carrying the CallError (a new ChannelOpenError { error: CallError } or equivalent), so the consumer branches on channel:open_failed + details.reason.

This is a breaking change to a method signature introduced in this crate's 0.4.x — acceptable at 0.5.0 (see Consequences), and it is the point of the change: the reason must be consumer-usable.

5. Compatibility and migration

• OpenHandler's type is unchanged. Existing registrations compile unchanged.
• ChannelCore::register_openable keeps its current signature (no establisher = always-OK); a new register_openable_with_establisher(spec, establisher, open_handler, registry, auth) variant adds the hook. alkhttp's OpenableAlpn gains an optional establisher field (default None) — the ferry passes it through mechanically.
• alktty migrates its channels path semantic failures (unknown backend, carriage != "raw", allocate_failed, ownership denial — currently post-open error frames) into the establisher, resolving them as channel:open_failed. Its direct-ALPN path keeps the in-band error frame (two transports, two contracts; the direct path has no open op to fail). The 0x00-peek disambiguation stays for the direct path only.
• alkhttp is unaffected (no openable ops in the default surface; the OpenableAlpn change is additive).

6. Panicked pump handlers stay EOF-shaped (pinned as designed)

The wrapper's teardown task swallows the pump handler's JoinError (let _ = raw_task.await). A panicked pump = instant EOF, which is the correct consumer-visible outcome for a mid-stream handler crash (indistinguishable from an abrupt close — there is no error channel mid-stream by design; establishment errors are the only kind that belong in the open reply). This ADR pins that as intended; no change. The establisher, by contrast, runs pre-reply — its panic (a future that panics when polled) surfaces as the spawned Once task's panic, which the serving loop already tolerates (the call never resolves; the deadline / client timeout is the bound). Establisher implementations return EstablishmentError instead of panicking, per this crate's no-panic convention.

Consequences

Positive:

• Establishment failure reaches the consumer as a typed, branchable call error — retry policy, client UX, and error reporting become possible for dial-refused, unknown-resource, and shortage cases (previously: instant-EOF ambiguity).
• The SSH contract ("the channel never exists opener-side") holds consumer-visibly: a failed open never returns a channel_id.
• No phantom channels: the ledger, policy count, and manager state are restored atomically on failure — allocation and teardown balance.
• alktty's per-crate in-band error vocabulary is retired on the channels path; every future ALPN crate (alktunnels first) gets the establishment reply for free.
• ADR-047 §3's "channel plan" shape is realized: awaited preparation before reply, spawned pumps after.

Negative:

• channel:open_failed + the reason vocabulary is a new wire-visible error surface — additive, but it joins the stable error set consumers may branch on (per ADR-016, details shapes are discoverable via services/schema).
• ChannelClient::open_channel's error type changes (breaking at 0.5.0; mechanical for consumers — the String was a debug-formatting wrapper anyway).
• OpenableAlpn (alkhttp) gains a field; its two construction sites add None (mechanical).
• The establisher await adds a bounded latency to open-op replies where handlers previously replied instantly (the spawn). The 10s default is the worst case for a hung establisher; real establishers (dial, lookup) complete in dial-time. Consumers already tolerate call-op latency; the deadline is the bound.

Door type

One-way (wire-visible error surface). channel:open_failed and its details.reason vocabulary join the stable error set: once consumers branch on reason codes, changing the vocabulary requires a migration (the same one-way-ness ADR-016 gives typed error details). The establisher hook shape itself — OpenEstablisher, the register_openable_with_establisher variant, the Establishment/EstablishmentError types — is a two-way-door implementation detail within the one-way decision (the wrapper shape, per ADR-047 §3's own door-type note). The OpenHandler type is untouched, which is what keeps the split cheap to revise.

Implementation units

1. alkcall 0.5.0 — OpenEstablisher + register_openable_with_establisher; wrapper flow (await bounded → teardown-on-failure → channel:open_failed with details); ChannelClient::open_channel typed error (N-1); tests:
   • establisher fails after allocation → consumer's open_channel resolves Err(channel:open_failed) + reason details; channel absent from channel_ids() afterward;
   • establisher never completes → timeout-reason failure within the deadline, channel torn down, ledger decremented;
   • no-establisher registration behaves exactly as today (compat gate);
   • establisher success spawns pumps and replies {channel_id} unchanged.
2. alktty migration — channels-path semantic failures move into an establisher; open_via_channels_surfaces_negotiation_rejected resolves via call error; the direct-ALPN error-frame path is retained.
3. alkhttp pass — OpenableAlpn.establisher: Option<...> (default None), threaded through the session fork (mechanical).

References

• Review 006 E-01 (the establishment gap — findings and prior-art survey), N-1 (the client error-type gap this ADR also resolves), E-03/E-04 (adjacent teardown/early-arrival notes, filed separately from this ADR's scope)
• ADR-047 §3 (openable ALPNs are operations — the "channel plan" wrapper shape this ADR restores; §7 opener ledger — the teardown un-increment path)
• ADR-016 (typed error schemas — the details vehicle)
• ADR-040/041 (backpressure/caps — untouched; the teardown path keeps the ledger take as the atomic gate)
• alktty ADR-009 (the open op's input is the negotiation) + review 001 L1/L3 — the in-band mechanism retired on the channels path
• alktunnels OQ-TN-09 (dial-failure reporting — the first consumer of the new error) and docs/research/ssh-socks5-survey.md §"Open-failure path" (the reason-code prior art)
• RFC 4254 §5.1, RFC 1928 §6 — SSH/SOCKS5 open-failure semantics

Amendment (Unit 1 implementation, 2026-09-06)

Unit 1 landed in alkcall 0.5.0. Two implementation-shape notes, both within this ADR's two-way door (the hook shape is the revisable implementation detail; the wire surface is unchanged from §1/§3):

1. The establisher does not receive the channel Connection. §1's signature sketch passed Connection to the establisher, but the channel's BiStream is yield-once (ChannelBidiStreamSource) — it cannot be handed to both the establisher and the pump handler, and the establisher is pre-data-plane by design (its dial targets the backend, not the channel). The implemented signature is Fn(Value, AuthContext) -> BoxFuture<'static, Result<Establishment, EstablishmentError>>; the Connection belongs exclusively to the OpenHandler (unchanged). Identity semantics (CF-005 corollary, 2026-09-07): the AuthContext the establisher and the pump handler receive is the per-call context — the opener's dispatch-resolved identity (the same identity the ACL gate and the per-identity cap check saw) overlaid onto the install-time context; transport-truthful fields (alpn, remote_addr, tls_client_fingerprint) stay install-time. On a per-connection registry (ADR-047 §4) this is identical to the install-time context; on hub-forwarded opens the establisher sees the end client, not the hub.
2. The bound is the earlier of the dispatch deadline and the per-registration timeout. §2 names the dispatch deadline "when the OperationContext carries one, else the crate constant"; implemented as min(deadline_remaining, timeout_override | ESTABLISHMENT_TIMEOUT) — the registration override stays meaningful for Query/Mutation-typed open ops (whose dispatch carries a 30s deadline; Sub clears it), and a deadline already in the past yields a zero bound (immediate timeout reason).

Implemented surface: OpenEstablisher, Establishment, EstablishmentError (reasons dial_failed/unknown_resource/ resource_shortage/handler_error + the wrapper's timeout), ESTABLISHMENT_TIMEOUT (10s), CHANNEL_OPEN_FAILED (channel:open_failed), ChannelCore::register_openable_with_establisher (register_openable delegates with establisher: None), ChannelOpenError (client-side typed error: CallFailed { error: CallError } / MissingChannelId / AdoptFailed, with call_error() + establishment_reason() accessors). All four verification gates from the review landed as tests (establisher failure e2e through a real channels connection with ledger un-increment + no-channel assertions, bounded timeout, no-establisher compat, establisher-success pump round-trip).

Amendment 2 (plan payload, 2026-09-07 — review 007 R-01/R-02)

Review 007 (from the alktunnels UDP POC) filed two follow-ups on the establishment surface; both landed in alkcall 0.6.0.

1. Establishment carries the channel plan (R-01). §1 reserved the payload ("today: nothing") and the wrapper consulted only success/failure — so an establisher whose backend produces a handle (a dialed socket, a TTY allocation) had to cross it to the pump handler through a per-crate side channel. The alktunnels POC shipped a resource-keyed slot + poll loop whose concurrent same-resource race is unfixable within that shape; alktty documented the same wall (backend allocate cannot cross, so failure classes stayed in-band — the phantom-channel shape ADR-049 removed, alive one layer down).

The plan is now real: Establishment { plan: Option<ChannelPlan> } with ChannelPlan = Arc<dyn Any + Send + Sync> — typed-opaque, not serde_json::Value. The review's Option<Value> sketch could not satisfy its own verification gate ("establisher dials, plan carries the handle"): the payloads establishers actually hand off are live handles with no JSON representation. The establisher and the OpenHandler agree on the concrete type; alkcall never inspects it. The wrapper threads establishment.plan to the handler's new second parameter (OpenHandler = Fn(Value, Option<ChannelPlan>, Connection, AuthContext) -> JoinHandle<()>); the separate-parameter shape wins over merging into input because a typed payload cannot ride the JSON input without a downcast-side registry and the reserved-key collision the review already anticipated. The plan is process-local (establisher → wrapper → handler on the producing side); the wire surface is unchanged — nothing crosses the transport that isn't already the open op's input. #[non_exhaustive] on Establishment keeps a future carrier change from being another breaking release. Construction is Establishment::new(plan) / Establishment::default(); the 0.5.0 Ok(Establishment {}) sites break mechanically at 0.6.0, which is the point of landing this now (before alktunnels Phase 1 ships the side-channel shape into a real crate and the payload lands later anyway as a second break).

2. The OpenHandler lifetime contract is documented (R-02). The wrapper awaits the returned JoinHandle and its completion triggers teardown — so the handle must track the data-plane lifetime: a handler that returns before its pumps finish tears the channel down at birth (the POC's first pump implementation hit exactly this: every tunnel connected then instantly EOF'd). The contract was implemented but never documented; the type docs now state it ("await the pumps inline, never spawn-and-forget and return early") on OpenHandler and the registration entry points, plus a debug! telemetry line in run_open_wrapper when a handler exits without having accepted the channel's BiStream (the birth-teardown hint; the accept is observable in-process via the yield-once source). §6's pinned EOF-shaped panic semantics are unchanged.

Amendment 3 (establisher reply projection, 2026-09-16 — review 008 U-2)

Amendment 2 filled the establisher → handler direction (the plan); the establisher → opener direction stayed empty — the wrapper hardcodes the success reply to channel_id (run_open_wrapper's ResponseEnvelope::ok). alktunnels ADR-008's bind-first listen establisher ends establishment at bind time, and the observed OS-chosen bound address must ride the open-op reply as an additive "bound" field (the SOCKS5 BIND reply#1 BND.ADDR fidelity ask) — the alternative (an out-of-band query op) is a new wire surface, strictly worse than an optional reply field.

The decision: Establishment gains optional reply fields the wrapper merges into the success output.

• The carrier: Establishment.reply_fields: Option<Map<String, Value>> (private, builder-constructed) — Establishment::new(plan).with_reply_field("bound", json!({...})), plus with_reply_fields(map) (batch) and reply_fields() (read). The map shape generalizes beyond bound without a fourth amendment; #[non_exhaustive] (amendment 2) made the carrier extension additive — no construction-site break.
• The merge: on establisher success the wrapper extends the success output with the contributed fields AFTER reserving channel_id. The channel_id reservation: the key is wrapper-owned; an establisher supplying it is an establisher bug — the wrapper fails the open loudly (channel:open_failed, reason handler_error, message naming the reserved key) and tears the just-allocated channel down, never silently shadowing its own value. Contributed fields otherwise merge flat; the output schema remains the op's own concern (the producing crate documents its optional fields — alktunnels' listen-op spec documents bound).
• The client read path: ChannelClient::open_channel_with_reply returns (channel_id, reply, send, recv) — the full success output — so a consumer reads bound without dropping to call_open_op + manual adopt_channel; open_channel delegates and discards the extra fields, signature unchanged.
• Compatibility: absent reply fields leave the reply byte-identical to the pre-amendment shape ({ channel_id }) — verified by test. Old consumers (open_channel extracting channel_id, ignoring unknown fields) are unaffected by a NEW field; new consumers reading bound against an OLD alkcall see the field absent — the additive posture alktunnels ADR-008 §2 pins.

Door type: the reply fields are an additive call-plane JSON surface (like channel:open_failed's details in §3 — consumers ignore unknown fields by the envelope's own parse posture); the field set an op may contribute is the producing crate's schema concern, not a protocol vocabulary. No data-plane change.

Implemented surface (alkcall 0.7.2): Establishment::with_reply_field / with_reply_fields / reply_fields, the wrapper's reservation check (RESERVED_REPLY_KEY = channel_id) + merge_reply_fields, ChannelClient::open_channel_with_reply. Verification gates landed as tests: the projection produces { channel_id, bound } on the wire; no-fields (establisher or not) produces exactly { channel_id } (byte-identical); the reserved-key establisher fails with reason handler_error, channel torn down, ledger decremented, handler never spawned; the e2e gate carries bound through a real channels connection to open_channel_with_reply while open_channel stays unchanged.