- open-questions.md: OQ-39 → ADR-052, OQ-40 → ADR-053 (resolutions replace the deferred-until-needed posture) - channel-operations.md: the put contract (initiator allocates + streams, reserved input key, one response, acks-in-result sentence), the control routing (ADR-052), the enumerator aggregation (ADR-053), the connection-owner paragraph's pub-era correction, the ADR table rows - ADR-051 §6: PubTypedOpen → the put translation; ADR-047's 2026-09-06 amendment records its era honestly (the dynamic half is now ADR-053) - review 010: all four findings resolved (with the S-01 sketch correction so the published record stops carrying the wrong-graph reading) - review 011: closed (all seven pins implemented + gates landed) - review 001 C-08: resolved note (odd/even + adopt, implemented) - AGENTS.md: the 'Stub Guard' section — the not_implemented grep is part of cargo test; the registration invariants and the implement-or-stop ruling are release-blocking Verified: cargo test (696+guard), clippy -D warnings, fmt, doc clean.
344 lines
18 KiB
Markdown
344 lines
18 KiB
Markdown
# ADR-051: In-Tree Channel Relay (`ChannelRelay`) and the Hub-Leg Assembly
|
||
|
||
## Status
|
||
|
||
Accepted (2026-09-17) — amends ADR-042 (the relay contract's
|
||
implementation home moves into alkcall; the translate-not-forward
|
||
contract is unchanged). Sequenced after review 008 remediation Units 1
|
||
and 2 (both landed, alkcall 0.8.0): the relay's establisher projects
|
||
the spoke reply's extra fields via `Establishment` reply fields
|
||
(ADR-049 amendment 3), and the spoke's flavor-form open ops arrive
|
||
reconstructed WITH the `channel_open` marker through discovery
|
||
(ADR-047 amendment 3). All three implementation units are landed:
|
||
Unit 3a (the `ChannelRelay` component, §1–§4, §6) in
|
||
`src/channels/relay.rs`, Unit 3b (the hub-leg install template, §5) in
|
||
`src/channels/hub_leg.rs`, and Unit 3c (the review's gate-2 e2e
|
||
harness, gates 6/7) in `src/channels/gate2_tests.rs` — alkcall 0.8.0.
|
||
|
||
## Context
|
||
|
||
Review 008's remediation plan (Unit 3) pinned the in-tree relay as the
|
||
implementation of the review's U-1 gate 2 — the hub-relay round trip
|
||
that proves flavor-form marked ops survive discovery into a genuine
|
||
relay. The as-filed ADR-042 §"Scope note" put the relay implementation
|
||
in `alknet-hub`/downstream; the consumers that now exist (alktunnels'
|
||
graduation, alkhttp's fallback hub, alknodes, and the hub/spoke family
|
||
behind them) all need the same shape, so the implementation moves into
|
||
alkcall as a reusable export.
|
||
|
||
The consumer family the relay must serve is the hub/spoke composition:
|
||
spokes connect to a hub and expose services; the hub re-exposes those
|
||
services to other spokes (and consumers) based on ACL. Both a hub and
|
||
a spoke can be a relay depending on the setup. A canonical example:
|
||
spoke A exposes `channels/tunnel/direct`; the hub imports it via
|
||
`from_call`, gates it with its own ACL, and re-exposes it on its
|
||
consumer-leg channel-0 registries; spoke B opens through the hub and
|
||
the hub relays the channel to spoke A. The hub performs no port
|
||
binding and no protocol work on the data plane — the property that
|
||
makes the hub-acl-relay role cheap (alktunnels' no-bind posture, the
|
||
same property ADR-042 §What the hub runs pins: "the hub never runs a
|
||
handler for `alk/tty`, `alk/ssh`, or `alk/tunnel`").
|
||
|
||
Two implementation surfaces were evaluated for the data plane; the
|
||
decision records why the wrapper-composition shape wins over
|
||
byte-forward-with-rewrite.
|
||
|
||
## Decision
|
||
|
||
### 1. The relay is the wrapper composition — translate-not-forward, implemented with the existing open-op machinery
|
||
|
||
ADR-042's two-layer contract is implemented as composition of pieces
|
||
this crate already has:
|
||
|
||
- **Layer 1 (call-protocol translate)** is the ADR-049 establisher:
|
||
the hub's consumer-leg open op resolves through the full open-op
|
||
wrapper (registry ACL → per-identity cap → allocation), then the
|
||
establisher calls the producer leg's open op, adopts the returned
|
||
spoke channel id into the producer-leg `ChannelManager`, and returns
|
||
`Establishment::new(plan)` whose plan carries the spoke-leg streams.
|
||
The hub allocates its own consumer-leg `channel_id`; the spoke
|
||
allocated its own producer-leg id; the two ids never appear in each
|
||
other's namespaces.
|
||
- **Layer 2 (byte-forward)** is the ADR-050 `OpenHandler`:
|
||
`pump_bidi(consumer_leg_bistream, spoke_recv, spoke_send)` — the
|
||
pump's generic bounds already match the relay's two legs. The relay
|
||
never parses chunk framing on either leg; the mux/demux on each leg
|
||
absorbs the 8-byte header (ADR-034) and the channel-id namespaces
|
||
are per-connection, so no header rewrite exists. Awaited inline per
|
||
the R-02 lifetime contract; on completion both legs tear down (the
|
||
wrapper's handler-exit teardown covers the consumer leg; the pump's
|
||
send/recv drops EOF the spoke leg).
|
||
|
||
**The relay map dissolves.** ADR-042 §`channel_id` mapping pinned a
|
||
`HashMap<channel_id, channel_id>` per (browser, spoke) pair maintained
|
||
by the hub. The implementation needs no map: each leg's `ChannelManager`
|
||
already holds its own id → routing state, and the per-channel pump
|
||
closure binds both legs' streams. The mapping is implicit-per-channel.
|
||
This is a two-way-door implementation detail within ADR-042's contract
|
||
(the ADR itself marked the mapping strategy two-way).
|
||
|
||
**`channel/close` and `channel/control` need no translation surface.**
|
||
Consumer-side close → wrapper teardown → pump abort → send-half drop →
|
||
EOF chunk → spoke handler exits → spoke's own wrapper decrements its
|
||
ledger. Close propagates through the EOF cascade with correct ledger
|
||
accounting on both legs; the hub never sees a `channel/close` to
|
||
translate. Control stays per-leg (`channel/control` is not implemented
|
||
upstream, OQ-39 — a hub wanting cross-leg control translation composes
|
||
it downstream). This supersedes the ADR-042 §mapping bullet's
|
||
close-translation sentence as an implementation matter: the mapping
|
||
entry the sentence's removal depended on does not exist.
|
||
|
||
### 2. What the relay holds on the producer leg: `CallConnection` + `ChannelManager`
|
||
|
||
The relay's producer-leg surface is `Arc<CallConnection>` (for the
|
||
translate hop's open call) plus the producer-leg `ChannelManager` (for
|
||
`adopt_channel` and teardown visibility) — not a `ChannelClient`.
|
||
|
||
Rationale: `ChannelClient`'s calling surface detaches on
|
||
`take_call_connection` (the hub needs the connection for `from_call`
|
||
import AND for the relay's establisher AND possibly its own ops —
|
||
three claimants), while `Arc<CallConnection>` is `Clone` and the
|
||
client's manager accessor yields the same `ChannelManager` the relay
|
||
needs. A hub that dials the spoke with `ChannelClient::from_connection`
|
||
and then tears the client down after import keeps the relay alive on
|
||
the `Arc`s. The establisher builds its forwarded payload directly
|
||
(`build_forwarded_payload` semantics — the hub as caller, the consumer
|
||
as `forwarded_for`, ADR-026 §3) rather than through
|
||
`call_open_op`, which carries no payload variant.
|
||
|
||
The spoke reply's `channel_id` is stripped before projection (the
|
||
consumer-leg reply carries the hub-allocated id, not the spoke's);
|
||
the spoke reply's other fields ride through `with_reply_fields` —
|
||
`bound` flows end-to-end with per-hop truthfulness (a hub re-producing
|
||
a listen resource binds its own listener and contributes its own
|
||
`bound`; alktunnels ADR-008 §Hub re-produce).
|
||
|
||
### 3. Reason-code mapping (spoke `CallError` → `EstablishmentError`)
|
||
|
||
The spoke's open can fail with the full call-error vocabulary; the
|
||
establisher maps it into `EstablishmentError` by class:
|
||
|
||
| Spoke failure | `EstablishmentError` variant | Wire reason |
|
||
|---|---|---|
|
||
| `channel:open_failed` with a typed reason | the same class — `DialFailed` for `dial_failed`/`timeout`, `UnknownResource` for `unknown_resource`, `ResourceShortage` for `resource_shortage`, `HandlerError` for `handler_error` | the mapped reason |
|
||
| `channel:open_failed` reason `timeout` | `DialFailed` (the spoke establishment exceeded the spoke bound — from the consumer's side it is unreachable-target-shaped) | `dial_failed`, spoke message preserved |
|
||
| `NOT_FOUND` (spoke predates the op) | `HandlerError` carrying the code | `handler_error` |
|
||
| `FORBIDDEN` (hub lacks the spoke grant) | `HandlerError` carrying the code | `handler_error` |
|
||
| other `CallError` | `HandlerError` carrying the code | `handler_error` |
|
||
|
||
The spoke's original code and message are preserved in the mapped
|
||
variant's `message` (never discarded — a consumer debugging a
|
||
FORBIDDEN-through-relay must see the spoke's grant failure, not a bare
|
||
`handler_error`). The `timeout` mapping is the one non-1:1 case: the
|
||
consumer-visible reason is `dial_failed` because establishment
|
||
timeouts are wrapper-generated (ADR-049 §3) and not an establisher
|
||
vocabulary word; the spoke's message carries the truth.
|
||
|
||
### 4. The registration seam: two phases, fork-registry registration
|
||
|
||
Openable ops register on a **per-connection fork of the hub's base
|
||
registry** installed as the consumer leg's dispatch registry (the
|
||
ADR-047 §4 amendment mechanism). Discovery of the producer leg's ops
|
||
happens once (hub startup or reconnect); registration happens per
|
||
consumer-leg connection inside the `install_channel_zero` hook. The
|
||
one-call signature the review sketched cannot exist without adapter
|
||
changes — the two phases are the API:
|
||
|
||
1. **Discover/stash** — `from_call` against the producer leg returns
|
||
the imported `HandlerRegistration` bundles. Marked specs (rebuilt
|
||
WITH the `channel_open` marker, ADR-047 amendment 3) are separated
|
||
from plain bundles by the marker. The stash is a `Clone`able
|
||
template: `HandlerRegistration` and `HandlerKind` are `Clone`, and
|
||
`OperationRegistry::fork` deep-copies registrations, so one
|
||
discovered set serves any number of consumer legs.
|
||
2. **Register per connection** — the hub-leg install hook forks the
|
||
base registry, registers the generic channel ops
|
||
(`ChannelOperations::register_on`), registers the stashed plain
|
||
bundles as-is (the existing forwarding handlers), and registers
|
||
each stashed marked spec via the relay's
|
||
`register_relay_openable` (the fork is the `consumer_registry`
|
||
argument), then dispatches channel 0 over the fork.
|
||
|
||
Per-consumer ACL differentiation is a composition consequence, not a
|
||
mechanism: the fork is per consumer leg, so a hub re-exposing
|
||
different op subsets to different consumers filters the stash per
|
||
fork. The imported spec's own `access_control` remains the gate for
|
||
the re-exposed op; the hub composes additional ACL at its assembly
|
||
layer.
|
||
|
||
### 5. The hub-leg install hook is an in-tree export (the call-half support)
|
||
|
||
Every existing test hand-rolls the hub-side install hook
|
||
(`make_install_channel_zero`-shaped); the hub/spoke family all need
|
||
the same composition. The relay lands with a documented hub-leg
|
||
template — fork, generic ops, bootstrap discovery, stashed bundles
|
||
(plain + relay), per-call identity resolution (`ServingConfig`-shaped),
|
||
single-stream dispatch loop — as an in-tree composition (unit-scoped:
|
||
Unit 3b). This is composition of existing pieces, not a new protocol
|
||
surface; its exact API shape is a two-way-door implementation detail.
|
||
The spoke side needs nothing new: a spoke serving ops through a hub is
|
||
the existing connect-side serving shape (`from_connection_with_serving`)
|
||
plus the producer-leg registration it already does.
|
||
|
||
**ACL layering note (pinned, because it surprises):** the spoke's
|
||
`AccessControl` sees only the hub's identity — the end consumer's
|
||
identity rides `forwarded_for` as metadata and is never consulted by
|
||
any `AccessControl::check` (ADR-024/ADR-026). "The hub re-exposes
|
||
based on ACL" therefore means: the hub-side gate is the imported
|
||
spec's ACL (+ hub policy) on the consumer leg; the spoke-side grant
|
||
goes to the hub identity, which needs scopes on every spoke op it
|
||
relays. The end consumer never authenticates to the spoke directly.
|
||
|
||
**Loud-install coverage note (review 009 C-1):** the template's four
|
||
install-failure arms all end the install task before the dispatch
|
||
loop (channel 0 never dispatches — never a silent stub). Two arms are
|
||
unit-test-pinned: the relay-openable registration failure (the
|
||
Pub-typed marked spec gate) and the plain-bundle registration failure
|
||
(an un-compilable schema — the registry's fail-closed rule). The
|
||
other two arms — generic channel-ops registration and bootstrap
|
||
discovery install — are best-effort-loud: both register
|
||
crate-internal specs whose schemas compile by construction, so
|
||
failure is not reachable through any public path and an injectable
|
||
failure seam would test the seam, not the arm. This note is the
|
||
contract: if a future change makes either arm reachable, it gets the
|
||
same pin.
|
||
|
||
### 6. Bounds: rejected shapes and residual notes
|
||
|
||
- **Pub-typed open ops** — ~~rejected loudly~~ **translated** (ADR-047
|
||
amendment 4 §4.7 — the put-forwarding translation: the hub allocates
|
||
its connect-side id, opens the spoke op in the background,
|
||
pumps the consumer bytes → spoke, and mirrors the spoke's single
|
||
response with its reserved key stripped; `PubTypedOpen` is removed).
|
||
The rejection remains for the genuinely-unsupported shapes: unmarked
|
||
specs (`UnmarkedSpec`) only.
|
||
- **Compounding establishment bounds** — each hop's establisher is
|
||
bounded independently (ADR-049 §2); an N-hop chain adds N bounds.
|
||
The consumer's dispatch deadline (`min(deadline, 10s)` per hop)
|
||
bounds each hop, not the chain. No fix — noted for capacity
|
||
planning.
|
||
- **Mid-establishment consumer disconnect** — the one teardown window
|
||
with a delay, with two reclaim signals pinned by the Unit 3c gate
|
||
(`gate2_mid_establishment_disconnect_reclaims_without_leak`): (a)
|
||
the consumer leg's just-allocated channel reclaims at the leg's own
|
||
transport EOF (`clear_all` + ledger decrement) — the hub has not
|
||
adopted, so the producer leg stays clean; (b) the spoke's channel —
|
||
the spoke allocates before its establisher replies — is the honest
|
||
residual: it cannot be reclaimed by the hub (no adoption exists),
|
||
so it lingers until a spoke-side teardown signal (the spoke↔hub leg
|
||
ending, or the spoke's own establishment bound). The gate bounds
|
||
the window deterministically by severing the spoke leg and asserts
|
||
the spoke-side reclaim + ledger decrement. A channel never outlives
|
||
its leg lifetimes; never a leak past the connection's lifetime.
|
||
- **Adopted-entry teardown on every plan path** — the hub's
|
||
producer-leg entry (the `adopt_channel` state) is owned by the
|
||
relay's `RelayPlan` via a `Drop` guard, so it reclaims whenever the
|
||
plan dies: the normal pump completion, the pump handler's
|
||
early-return arms (plan absent, downcast failure, `try_unwrap`
|
||
failure, `accept_bi` failure), and the wrapper's establishment
|
||
bound expiring after the adopt (the plan dropped before the pump
|
||
ever spawns). Dropping the plan's send half also EOFs the spoke
|
||
side through the mux pump's implicit-EOF sentinel (REQ-CH-01), so
|
||
the spoke handler reclaims through the same cascade the normal
|
||
path uses. This closes the adopt-then-timeout window the landing
|
||
review surfaced (the §6 window above is the *pre-adopt* window;
|
||
this is the *post-adopt* one, now closed by construction). The
|
||
one remaining cancellation point *inside* `adopt_channel`'s own
|
||
await is the same pre-existing client-side window
|
||
(`ChannelClient::open_channel`'s adopt has it too) — bounded by
|
||
the leg's transport-EOF `clear_all`, not by the plan.
|
||
- **Multi-hop relays** — a chain of relays composes (each hop is a
|
||
hub leg pair), with the compounding-bound note above. No special
|
||
machinery.
|
||
|
||
## Consequences
|
||
|
||
**Positive:**
|
||
|
||
- The relay is O(1) new machinery: an establisher closure, a
|
||
`pump_bidi` handler, and a registration entry point. No parallel
|
||
auth path (the consumer leg's full wrapper machinery applies), no
|
||
new wire surface (the relay speaks only already-stable ops), no
|
||
data-plane parsing (ADR-034's one-way door stays closed).
|
||
- `from_call` gains a real in-tree consumer for the marker-branching
|
||
seam Gap C named (ADR-047 §Gap C): downstream crates compose
|
||
`ChannelRelay` instead of re-deriving the hub shape.
|
||
- U-1's gate 2 (the review's silent-failure scenario — flavor-form ops
|
||
relaying as plain forwarding stubs) becomes a permanent e2e gate.
|
||
**Negative:**
|
||
|
||
- New pub API (`ChannelRelay`, the registration entry point, the
|
||
hub-leg template) — minor bump, one release with the review's other
|
||
units, review-007 precedent.
|
||
- The two-phase discover/stash-then-register shape is more assembly
|
||
code for a trivial hub than a one-call import would be. The fork
|
||
mechanism forces it; the stash being `Clone` keeps the per-connection
|
||
work cheap.
|
||
- The hub holds `Arc<CallConnection>` + `ChannelManager` per spoke leg
|
||
for the connection's lifetime — per-leg state, bounded by the spoke
|
||
count.
|
||
|
||
## Door type
|
||
|
||
**Two-way (implementation shape) within a one-way posture.** The
|
||
translate-not-forward contract (ADR-042) and the ACL layering are
|
||
unchanged one-way decisions; the in-tree export, the implicit-mapping
|
||
implementation, and the two-phase registration shape are revisable
|
||
implementation details — downstream crates compose the export, not the
|
||
internals. The rejection posture for Pub-typed marked ops is loud (an
|
||
assembly error), so a future implementation of the Pub path is
|
||
additive.
|
||
|
||
## Verification gates (implementation units)
|
||
|
||
Unit 3a (the relay component):
|
||
|
||
1. `register_relay_openable` on a forked consumer-leg registry
|
||
resolves the open op end-to-end; the establisher populates
|
||
`forwarded_for` from the consumer's per-call identity (ADR-026 §3)
|
||
and strips the spoke reply's `channel_id`.
|
||
2. A spoke `channel:open_failed` maps to the consumer-leg
|
||
`channel:open_failed` with the mapped reason (the §3 table) and the
|
||
spoke's message preserved; the consumer leg tears down (ledger
|
||
decremented); no spoke channel leaks.
|
||
3. `bound` (Unit 1's reply projection) survives the relay to the
|
||
consumer reply.
|
||
4. Pub-typed marked specs are rejected at registration.
|
||
|
||
Unit 3b (the hub-leg template):
|
||
|
||
5. The template composes fork + generic ops + bootstrap discovery +
|
||
plain bundles + relay openables + serving identity and dispatches
|
||
channel 0; `services/list` on the consumer leg shows the
|
||
re-exposed ops.
|
||
|
||
Unit 3c (the gate-2 e2e):
|
||
|
||
6. The review's full harness: consumer → hub (forked channel-0
|
||
registry + relay) → producer, opening `channels/tunnel/direct` end
|
||
to end: the open resolves with the hub-allocated `channel_id`, the
|
||
establisher-contributed `bound` field survives to the consumer
|
||
reply, data flows both directions (the hub never parses it), and
|
||
hub-side disconnect / spoke-side close tears down both legs
|
||
(including the mid-establishment window from §6).
|
||
7. A companion `channels/tty/sub` relay run pins no regression on
|
||
standard shapes.
|
||
|
||
## References
|
||
|
||
- ADR-042 (amended by this ADR — the implementation home; the
|
||
contract unchanged; the relay map and close-translation bullets
|
||
superseded by the implicit-per-channel shape)
|
||
- ADR-047 amendment 3 (flavor-form ops reconstruct WITH the marker —
|
||
the discovery input this relay consumes), §Gap C (the
|
||
marker-branching seam), §4 amendment (the fork mechanism Unit 3b
|
||
composes)
|
||
- ADR-049 amendments 2/3 (the plan payload the relay's establisher
|
||
carries; the reply projection `bound` rides through)
|
||
- ADR-026 §3 (forwarded_for on the translate hop), ADR-050
|
||
(`pump_bidi` — the byte-forward hop), ADR-040/041 (the caps the
|
||
consumer leg keeps enforcing)
|
||
- alktunnels ADR-007/008 (the first consumers: the direct op's relay
|
||
and the listen `bound` field's per-hop truthfulness)
|
||
- Review 008 remediation plan Units 1–3 (the sequencing this ADR
|
||
lands inside) |