docs(adr): 094 — per-identity channel cap as DoS defense
ADR-076 framed the per-connection max_channels=256 cap as the DoS defense, but a peer can open an unbounded number of transport connections, so a per-connection cap bounds a connection's reassembly-buffer cost, not a peer's total channels. The only coherent unit for a channel DoS defense is the identity. ADR-094 records the corrected design: a ChannelLifecyclePolicy trait in channels-call (where the identity is already on OperationContext), consulted by the channel/open handler (after AccessControl::check, before allocation) and the channel/close handler (after the drain completes). Default is PerIdentityChannelPolicy::new(256) — 256 per PeerId across all the peer's connections, shared via Arc across every channels connection a peer accepts. The cap is a peer concern (not hub-specific), symmetric (both sides enforce), and lives in channels-call because the channels layer is auth-blind by design (ADR-075) — that is what makes it WASM-compatible, transport-agnostic, and ALPN-blind. For the hub-relay path (ADR-079), the spoke sees the hub as the direct caller (ADR-032 — forwarded_for is metadata, not authority, for the cap as for AccessControl::check), so the spoke caps the hub, not the browser. A spoke serving a high-fan-out hub sets the hub peer's cap higher via with_per_identity_caps — the spoke's own policy, not the hub's. Recursive channels do not bypass the cap (the same policy can be wired into the inner ChannelOperations). ADR-076 is amended: the per-connection max_channels is reframed as a per-connection memory bound (still returns channel:too_many_channels when hit), the "DoS defense summary" table is removed, and the "per-connection, not per-peer" line (the channels layer confessing a hole and hoping the layer above would fill it) is corrected. Spec docs updated to reference ADR-094: channel-operations.md gains a "Per-identity channel cap" section (trait, default, enforcement point, relay consequence, recursion); channels-adapter.md adds the policy check as step 3 of the channel/open handler and the decrement in channel/close; hub README adds the channel_policy field on Hub, the with_channel_policy builder, a dedicated subsection, and the inbound-peer-vs-hub-as-caller distinction; channels/README.md adds ADR-094 to the Applicable ADRs table and a 9th Key Design Principle; docs/architecture/README.md adds a Current State note and the ADR table row.
This commit is contained in:
glm-5.2 committed
1 parent
762d9c7bd2
commit
0fcd5bc322
7 files changed
+789
-51
No files matched your search
@@ -37,7 +37,8 @@ handler owns its sub-stream multiplexing on the `BiStream` it receives.
|
||||
| [073](../../decisions/073-channel-lifecycle-operations.md) | Channel Lifecycle Operations on the Call Protocol | `channel/open`/`close`/`control`/`resources/subscribe`; `direction` semantics; subscribe not poll |
|
||||
| [074](../../decisions/074-channelconnection-bidistreamsource.md) | ChannelConnection — BidiStreamSource over Chunk Reassembly | Per-channel `BidiStreamSource` impl; `accept_bi` yields `BiStream` (amended by ADR-093 — `into_sub_streams` removed) |
|
||||
| [075](../../decisions/075-channelsadapter-and-channelmanager.md) | ChannelsAdapter and ChannelManager | Substrate-agnostic demux loop; REQ-CH-01..04 contracts |
|
||||
| [076](../../decisions/076-backpressure-channel-limits-id-reuse.md) | Backpressure, Channel Limits, and ID Reuse | Bounded-buffer (1 MiB default), 256-channel cap, monotonic IDs with wrap |
|
||||
| [076](../../decisions/076-backpressure-channel-limits-id-reuse.md) | Backpressure, Channel Limits, and ID Reuse | Bounded-buffer (1 MiB default), 256-channel per-connection memory bound, monotonic IDs with wrap (DoS defense reframed by ADR-094) |
|
||||
| [094](../../decisions/094-per-identity-channel-cap.md) | Per-Identity Channel Cap | 256 per `PeerId`, enforced via `ChannelLifecyclePolicy` in `channels-call`; per-connection `max_channels` reframed as a memory bound; symmetric (both sides enforce); spoke caps hub (direct caller), not browser (forwarded_for is metadata) |
|
||||
| [077](../../decisions/077-tty-inside-channels.md) | TTY Inside Channels — Sub-Streams, Not Wire Format | TTY's two modes (direct vs channels); TTY always uses its 5-byte format, carried transparently in the channels payload |
|
||||
| [078](../../decisions/078-two-pump-shutdown-on-completion.md) | Two-Pump Shutdown-on-Completion Pattern | The two-pump deadlock contract; handler-level, not channels-layer |
|
||||
| [079](../../decisions/079-hub-relay-translate-not-forward.md) | Hub Relay — Translate, Not Transparently Forward | The hub translates channel 0, byte-forwards data channels with ID rewrite |
|
||||
@@ -120,6 +121,19 @@ handler owns its sub-stream multiplexing on the `BiStream` it receives.
|
||||
channels are byte-forwarded with `channel_id` rewrite (a 4-byte rewrite
|
||||
within the 8-byte header). This preserves the auth model. See ADR-079.
|
||||
|
||||
9. **The channel cap is per-identity, not per-connection.** A channel
|
||||
slot is a resource; the cap on how many an identity may hold open is
|
||||
a quota check, parallel to `OwnershipProvider::owns` (ADR-050) for
|
||||
spawned resources. The cap lives in `channels-call` (the channels
|
||||
layer is auth-blind by ADR-075 — no identity, no scopes), consulted
|
||||
by the `channel/open` and `channel/close` handlers after
|
||||
`AccessControl::check`. The default is `PerIdentityChannelPolicy::
|
||||
new(256)` — 256 per `PeerId` across all the peer's connections. The
|
||||
per-connection `max_channels` (ADR-076) is a memory bound, not a
|
||||
DoS defense. The cap is symmetric (both sides enforce); the spoke
|
||||
caps the hub as direct caller, not the browser as `forwarded_for`
|
||||
(metadata, not authority — ADR-032). See ADR-094.
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/alknet-channels/phase-0-findings.md` — Phase 0 research
|
||||
|
||||
@@ -235,6 +235,150 @@ The hub ran **zero** protocol-specific auth. It ran `channel/open`'s
|
||||
`AccessControl::check` (call-protocol machinery) and forwarded. The channels
|
||||
layer inherited the auth model by being a call-protocol operation.
|
||||
|
||||
## Per-identity channel cap (ADR-094)
|
||||
|
||||
A channel slot is a resource. The cap on how many channels an identity
|
||||
may hold open is a quota check on that resource — parallel to
|
||||
`OwnershipProvider::owns` (ADR-050) for spawned resources. Same
|
||||
primitive, different resource. The cap is a **peer concern**, not a
|
||||
hub-specific concern: any accepting peer (worker or hub) enforces the
|
||||
cap on its inbound channels, just as it enforces `AccessControl::check`
|
||||
on `channel/open`. The cap is also **symmetric** — both sides of a
|
||||
channels connection enforce their cap on the other's channels.
|
||||
|
||||
### Why the cap is not in the channels layer
|
||||
|
||||
`ChannelManager` (ADR-075) is auth-blind by design — no auth state, no
|
||||
identity, no scopes. That decision is load-bearing (it is what makes
|
||||
the channels layer WASM-compatible, transport-agnostic, and
|
||||
ALPN-blind). So the per-identity cap lives in `channels-call`, where
|
||||
the identity is already on `OperationContext` (the same place
|
||||
`AccessControl::check` runs). The channels layer (`channels-core`) is
|
||||
unchanged. See ADR-094 §"Why the channels layer cannot hold the cap".
|
||||
|
||||
The channels-layer per-connection `max_channels = 256` (ADR-076) is
|
||||
a **per-connection memory bound** (limits one connection's
|
||||
reassembly-buffer cost), not a DoS defense. A peer can open an
|
||||
unbounded number of transport connections, so a per-connection cap is
|
||||
not a per-peer DoS defense. The per-identity DoS defense is the cap
|
||||
documented here; see ADR-094 for the corrected DoS-defense framing.
|
||||
|
||||
### The `ChannelLifecyclePolicy` trait
|
||||
|
||||
```rust
|
||||
/// Per-identity channel lifecycle policy. Consulted by the
|
||||
/// `channel/open` handler (after `AccessControl::check`, before
|
||||
/// allocation) and the `channel/close` handler (after deallocation).
|
||||
/// Both handlers have the identity via `OperationContext`.
|
||||
pub trait ChannelLifecyclePolicy: Send + Sync + 'static {
|
||||
/// Before channel allocation. Deny with `channel:too_many_channels`
|
||||
/// (ADR-073) when the identity is over its cap. The identity is
|
||||
/// the direct caller (the peer that opened this channels
|
||||
/// connection); `forwarded_for` is metadata and is NOT consulted
|
||||
/// (ADR-032).
|
||||
fn check_open(&self, identity: &Identity) -> Result<(), ChannelError>;
|
||||
|
||||
/// After channel deallocation. Decrement the per-identity count.
|
||||
/// Called by the `channel/close` handler after the drain completes
|
||||
/// (ADR-076 §channel-id-reuse).
|
||||
fn on_close(&self, identity: &Identity);
|
||||
}
|
||||
```
|
||||
|
||||
### Default: `PerIdentityChannelPolicy::new(256)`
|
||||
|
||||
The default constructor enforces 256 per identity out of the box — no
|
||||
"NoOp default + wire it later." A channels-accepting peer that
|
||||
constructs `ChannelOperations::new(manager)` with no policy argument
|
||||
gets `PerIdentityChannelPolicy::new(256)`. The default is secure;
|
||||
opt-outs are explicit:
|
||||
|
||||
- `PerIdentityChannelPolicy::new(cap)` — shared per-identity state
|
||||
(`HashMap<PeerId, usize>` + cap), constructed **once per accepting
|
||||
peer** and shared (via `Arc`) across every channels connection that
|
||||
peer accepts. The sharing is what makes the cap per-identity, not
|
||||
per-connection.
|
||||
- `PerIdentityChannelPolicy::with_per_identity_caps(mapping)` —
|
||||
per-peer-role variant: `HashMap<PeerId, usize>` overrides the
|
||||
default cap for specific peers. Used by a spoke that serves a
|
||||
high-fan-out hub (the hub peer's cap is set higher than a worker
|
||||
peer's cap — see "Relay consequence" below).
|
||||
- `NoCap` — no cap. Explicit opt-out for tests, POCs, and trusted
|
||||
single-peer deployments. Not the default.
|
||||
|
||||
The policy is constructed once and passed to `ChannelOperations` at
|
||||
registration time:
|
||||
|
||||
```rust
|
||||
let policy = Arc::new(PerIdentityChannelPolicy::new(256));
|
||||
let channel_ops = ChannelOperations::new(manager, policy);
|
||||
channel_ops.register_on(&mut call_registry)?;
|
||||
```
|
||||
|
||||
### Enforcement point: between `AccessControl::check` and allocation
|
||||
|
||||
The `channel/open` handler (above) gains the policy check after ACL
|
||||
and before `next_id.fetch_add`:
|
||||
|
||||
1. ACL is already checked by `OperationRegistry::invoke` (the existing
|
||||
`AccessControl::check` path — unchanged).
|
||||
2. **NEW:** `policy.check_open(&op_ctx.identity)?` — deny with
|
||||
`channel:too_many_channels` if over cap.
|
||||
3. Allocate the `channel_id` via `next_id.fetch_add(1, Relaxed)`
|
||||
(DP-1: server-assigned — unchanged).
|
||||
4. Construct the `ChannelBidiStreamSource`, spawn the handler, record
|
||||
the `ChannelState` (unchanged).
|
||||
5. Return the `channel_id`.
|
||||
|
||||
The `channel/close` handler gains the decrement after the drain
|
||||
completes (the same point ADR-076 marks the `channel_id` as eligible
|
||||
for reuse):
|
||||
|
||||
1. Drain the reassembly buffer for `channel_id` (existing — ADR-076
|
||||
§channel-id-reuse).
|
||||
2. **NEW:** `policy.on_close(&op_ctx.identity)` — decrement the
|
||||
per-identity count.
|
||||
3. Return `{ "closed": true }` (unchanged).
|
||||
|
||||
### Relay consequence: the spoke caps the hub, not the browser
|
||||
|
||||
When the hub relays a browser's channel to a spoke (ADR-079), the
|
||||
spoke sees the hub as the direct caller. `forwarded_for` carries the
|
||||
browser's identity as metadata (ADR-032 — `forwarded_for` is not
|
||||
authority; `AccessControl::check` never reads it). The channel cap
|
||||
follows the same shape: the spoke's `ChannelLifecyclePolicy` is
|
||||
consulted with the **hub's** identity, not the browser's. The spoke
|
||||
asks "does the hub have access to open another channel?" and the
|
||||
hub's quota on the spoke reflects the aggregate of all relayed
|
||||
channels. The hub's per-browser caps are the hub's own concern
|
||||
(enforced on the browser leg by the hub's own policy), not the
|
||||
spoke's.
|
||||
|
||||
This is correct and consistent — the spoke authorizes the hub for
|
||||
container access the same way it authorizes any peer, and the hub's
|
||||
browser-relay ACL is the hub's own layer. The channel cap follows the
|
||||
same pattern as any other resource ACL.
|
||||
|
||||
**Deployment consequence:** a spoke that serves a hub relaying for
|
||||
many browsers must set the hub peer's cap higher than a worker peer's
|
||||
cap, or the spoke denies legitimate relayed channels when the hub's
|
||||
aggregate count exceeds a worker-sized cap. This is a per-peer-role
|
||||
policy, set by the spoke via `with_per_identity_caps`. The
|
||||
architecture provides the mechanism; the deployment sets the numbers.
|
||||
This is not a flaw — it is the same shape as any per-peer ACL (a
|
||||
spoke may authorize one peer for 1000 containers and another for 10;
|
||||
the channel cap is the same kind of per-peer policy).
|
||||
|
||||
### Recursive channels do not bypass the cap
|
||||
|
||||
A recursive `alknet/channels`-inside-`alknet/channels` channel runs a
|
||||
new `ChannelsAdapter` with a new `ChannelManager`. If the same
|
||||
`ChannelLifecyclePolicy` is wired into the inner `ChannelOperations`,
|
||||
the inner channels are counted against the same identity. Recursion
|
||||
is not a bypass; the 13-byte-per-chunk overhead is the documented
|
||||
cost (ADR-093), and the cap behavior is unchanged. Recursive channels
|
||||
are an edge case for edge cases and not specced further.
|
||||
|
||||
## Hub relay contract (ADR-079 — summary)
|
||||
|
||||
The hub **translates**, not transparently forwards:
|
||||
@@ -267,14 +411,17 @@ All design decisions are documented as ADRs in [decisions/](../../decisions/).
|
||||
| [073](../../decisions/073-channel-lifecycle-operations.md) | Channel Lifecycle Operations | The four ops; `direction` pinned; subscribe not poll |
|
||||
| [072](../../decisions/072-channel-0-pre-negotiated-call.md) | Channel 0 Pre-Negotiated | Channel 0 = `alknet/call` |
|
||||
| [079](../../decisions/079-hub-relay-translate-not-forward.md) | Hub Relay | Translate channel 0, byte-forward data channels |
|
||||
| [094](../../decisions/094-per-identity-channel-cap.md) | Per-Identity Channel Cap | 256 per `PeerId`, enforced via `ChannelLifecyclePolicy` in `channels-call`; per-connection `max_channels` reframed as a memory bound |
|
||||
| [093](../../decisions/093-channels-pure-channel-multiplexing.md) | channels Pure Channel Multiplexing | No `stream_types` on `channel/open`; no `stream_type` on `channel/control`; handler owns sub-stream multiplexing |
|
||||
| [049](../../decisions/049-streaming-handler-for-subscriptions.md) | StreamingHandler | The machinery `channel/resources/subscribe` uses |
|
||||
| [032](../../decisions/032-forwarded-for-identity.md) | Forwarded-For Identity | The auth chain for hub-relayed opens |
|
||||
| [050](../../decisions/050-dynamic-resource-ownership-for-runtime-spawned-resources.md) | Dynamic Resource Ownership | The ownership store the spoke queries |
|
||||
| [032](../../decisions/032-forwarded-for-identity.md) | Forwarded-For Identity | The auth chain for hub-relayed opens (and why the cap is per direct-caller, not per `forwarded_for`) |
|
||||
| [050](../../decisions/050-dynamic-resource-ownership-for-runtime-spawned-resources.md) | Dynamic Resource Ownership | The parallel — a channel slot is a resource, the cap is a quota check |
|
||||
|
||||
## References
|
||||
|
||||
- ADR-073: channel lifecycle operations (the decision)
|
||||
- ADR-094: per-identity channel cap (the cap, the trait, the relay
|
||||
consequence)
|
||||
- ADR-079: hub relay (the translate contract)
|
||||
- `docs/research/alknet-channels/phase-0-findings.md` §Channel Open
|
||||
Negotiation, §ACL and Security Model
|
||||
@@ -78,7 +78,14 @@ pub struct ChannelManager {
|
||||
// call-protocol-blind.
|
||||
next_id: AtomicU32, // monotonic; wraps at u32::MAX
|
||||
buffer_cap: usize, // default 1 MiB (ADR-076)
|
||||
max_channels: usize, // default 256 (ADR-076)
|
||||
max_channels: usize, // default 256 (ADR-076) — per-connection
|
||||
// memory bound, NOT a DoS defense. The
|
||||
// per-identity DoS defense is the
|
||||
// ChannelLifecyclePolicy consulted by the
|
||||
// channel/open handler in channels-call
|
||||
// (ADR-094). The auth-blindness that forces
|
||||
// the cap out of this struct is ADR-075's
|
||||
// "no auth state" rule.
|
||||
}
|
||||
|
||||
struct ChannelState {
|
||||
@@ -136,27 +143,55 @@ dependencies.
|
||||
|
||||
The `channel/open` (and `channel/close`, `channel/control`,
|
||||
`channel/resources/subscribe`) operations are registered on the call
|
||||
protocol's `OperationRegistry` at assembly time:
|
||||
protocol's `OperationRegistry` at registration time. The
|
||||
`ChannelOperations` constructor takes a `ChannelLifecyclePolicy`
|
||||
(ADR-094) — the default is `PerIdentityChannelPolicy::new(256)` (a
|
||||
real per-identity cap, not NoOp):
|
||||
|
||||
```rust
|
||||
let channel_ops = ChannelOperations::new(manager.clone());
|
||||
let policy = Arc::new(PerIdentityChannelPolicy::new(256));
|
||||
let channel_ops = ChannelOperations::new(manager.clone(), policy);
|
||||
channel_ops.register_on(&mut call_registry)?;
|
||||
```
|
||||
|
||||
The same `Arc<PerIdentityChannelPolicy>` is shared across every
|
||||
channels connection this peer accepts — that is what makes the cap
|
||||
per-identity, not per-connection. A hub constructs one policy and
|
||||
shares it across all worker and browser legs; a worker accepting
|
||||
direct channels constructs one policy and shares it across whatever
|
||||
connections it accepts. See ADR-094 for the policy trait and the
|
||||
default/opt-out variants.
|
||||
|
||||
The `channel/open` handler (ADR-073):
|
||||
1. ACL is already checked by `OperationRegistry::invoke` before this handler
|
||||
runs.
|
||||
2. Looks up the ALPN in `HandlerRegistry` → `channel:unknown_alpn` if
|
||||
missing.
|
||||
3. Allocates the `channel_id` via `next_id.fetch_add(1, Relaxed)` (DP-1:
|
||||
server-assigned).
|
||||
4. Constructs the `ChannelBidiStreamSource` (ADR-074, as amended by
|
||||
3. **Per-identity cap check (ADR-094):**
|
||||
`policy.check_open(&op_ctx.identity)?` — deny with
|
||||
`channel:too_many_channels` if the identity is over its cap. The
|
||||
identity is the direct caller (the peer on this channels
|
||||
connection); `forwarded_for` is metadata and is NOT consulted
|
||||
(ADR-032). For the hub-relay path, the spoke sees the hub as the
|
||||
direct caller — the hub's quota on the spoke reflects the aggregate
|
||||
of all relayed channels (ADR-094 §5).
|
||||
4. Allocates the `channel_id` via `next_id.fetch_add(1, Relaxed)` (DP-1:
|
||||
server-assigned). The per-connection `max_channels` (ADR-076) is
|
||||
checked here too — the per-connection memory bound; if hit, the same
|
||||
`channel:too_many_channels` error is returned (which cap fired first
|
||||
is an implementation detail — ADR-094 §4).
|
||||
5. Constructs the `ChannelBidiStreamSource` (ADR-074, as amended by
|
||||
ADR-093) — one reassembly buffer, yielding a `BiStream`.
|
||||
5. Spawns the handler task — `tokio::spawn(handler.handle(conn, &auth))`.
|
||||
6. Spawns the handler task — `tokio::spawn(handler.handle(conn, &auth))`.
|
||||
Identical to what `TtyAdapter::handle` does today, but on a
|
||||
channels-backed `Connection`.
|
||||
6. Records the `ChannelState`.
|
||||
7. Returns the `channel_id`.
|
||||
7. Records the `ChannelState`.
|
||||
8. Returns the `channel_id`.
|
||||
|
||||
The `channel/close` handler (ADR-073) gains a symmetric
|
||||
`policy.on_close(&op_ctx.identity)` call after the drain completes
|
||||
(the same point ADR-076 marks the `channel_id` as eligible for reuse)
|
||||
— decrementing the per-identity count.
|
||||
|
||||
## Demux invariants (REQ-CH-02, 04)
|
||||
|
||||
@@ -258,7 +293,8 @@ All design decisions are documented as ADRs in [decisions/](../../decisions/).
|
||||
|-----|----------|---------|
|
||||
| [075](../../decisions/075-channelsadapter-and-channelmanager.md) | ChannelsAdapter and ChannelManager | The split; the contracts |
|
||||
| [093](../../decisions/093-channels-pure-channel-multiplexing.md) | channels Pure Channel Multiplexing | The umbrella decision: 8-byte header, no `stream_type`, one reassembly buffer per channel |
|
||||
| [076](../../decisions/076-backpressure-channel-limits-id-reuse.md) | Backpressure, Limits, ID Reuse | Bounded-buffer, 256-channel cap, monotonic IDs |
|
||||
| [076](../../decisions/076-backpressure-channel-limits-id-reuse.md) | Backpressure, Limits, ID Reuse | Bounded-buffer, 256-channel per-connection memory bound, monotonic IDs (DoS defense reframed by ADR-094) |
|
||||
| [094](../../decisions/094-per-identity-channel-cap.md) | Per-Identity Channel Cap | 256 per `PeerId`, enforced via `ChannelLifecyclePolicy` in `channels-call`; per-connection `max_channels` reframed as a memory bound |
|
||||
| [078](../../decisions/078-two-pump-shutdown-on-completion.md) | Two-Pump Pattern | Shutdown-on-completion contract |
|
||||
| [079](../../decisions/079-hub-relay-translate-not-forward.md) | Hub Relay | Translate channel 0, byte-forward data channels |
|
||||
|
||||
@@ -271,7 +307,11 @@ All design decisions are documented as ADRs in [decisions/](../../decisions/).
|
||||
- ADR-073: channel lifecycle operations (the ops registered on `call_ops`)
|
||||
- ADR-074: ChannelBidiStreamSource (what the manager constructs per
|
||||
channel, as amended by ADR-093)
|
||||
- ADR-076: backpressure and limits (`buffer_cap`, `max_channels`)
|
||||
- ADR-076: backpressure and limits (`buffer_cap`, `max_channels` — the
|
||||
per-connection memory bound)
|
||||
- ADR-094: per-identity channel cap (the `ChannelLifecyclePolicy`
|
||||
consulted by the `channel/open` handler; the relay consequence for
|
||||
hub-relayed channels)
|
||||
- `docs/research/alknet-channels/poc-summary.md` §Issues Surfaced #4-#7
|
||||
(REQ-CH-01..04, the two-pump deadlock)
|
||||
- `docs/research/stream-unification/findings.md` — the research that
|
||||
|
||||
Reference in new issue
Block a user