glm-5.1
bc0c2589c7
- Resolve OQ1: WS server accepts raw WebSocket instances via addConnection/removeConnection (framework-agnostic, not coupled to Hono/Express/Bun/Deno) - Resolve OQ2: Backpressure handled by disconnecting slow consumers at configurable threshold (default 1MB), with onBackpressure callback for observability - Resolve OQ3: Topic-based fan-out with subscription tracking instead of broadcast-all; spokes send __subscribe/__unsubscribe control events; direct messaging via 'direct:' topic pattern Add ADR-003 for subscription control protocol decision. Update all fan-out adapters (WS server, Iroh hub) and spoke adapters (WS client, Iroh spoke) with subscription tracking/forwarding. Fix routing key ambiguity (full topic string, not event type alone). Add error handling, composition, and reserved type sections. Clarify Worker as symmetric-only.
65 lines
4.3 KiB
Markdown
65 lines
4.3 KiB
Markdown
# ADR-003: Subscription Control Protocol for Fan-Out Adapters
|
|
|
|
**Status**: Accepted
|
|
**Date**: 2026-05-08
|
|
|
|
## Context
|
|
|
|
Fan-out adapters (WebSocket server, Iroh hub, Worker pool) send events to multiple connected clients. A naive implementation broadcasts every event to all connections. This wastes bandwidth and — for many use cases — leaks data to clients that shouldn't receive it (e.g., a chat room message sent to clients not in that room).
|
|
|
|
The `type:id` topic scoping pattern already provides the right routing abstraction (`"message.sent:conv-123"` is a natural fan-out key). But the server side needs to know which clients are interested in which topics. This requires a subscription control protocol: spokes must tell the hub what they want to receive.
|
|
|
|
Two design options were considered:
|
|
|
|
1. **Reserved envelope types** — use the existing `EventEnvelope` format with `__`-prefixed types for control messages (`__subscribe`, `__unsubscribe`)
|
|
2. **Separate control channel** — a distinct message schema for control, separate from data messages
|
|
|
|
## Decision
|
|
|
|
Use reserved envelope types (`__subscribe` / `__unsubscribe`) within the existing `EventEnvelope` format.
|
|
|
|
Control events:
|
|
|
|
```json
|
|
{ "type": "__subscribe", "id": "", "payload": { "topic": "message.sent:conv-123" } }
|
|
{ "type": "__unsubscribe", "id": "", "payload": { "topic": "message.sent:conv-123" } }
|
|
```
|
|
|
|
Convention: event types starting with `__` are reserved for adapter control. They are handled internally and never dispatched to user-facing listeners. The `id` field in control events is the empty string `""` by convention — control events use the `topic` field in their `payload` for routing instead of the `type:id` scoping pattern.
|
|
|
|
## Rationale
|
|
|
|
1. **Single message format** — one serialization format between spoke and hub. No need to distinguish "is this a control message or a data message?" before parsing. Both are valid `EventEnvelope` objects.
|
|
|
|
2. **Consistent with Redis model** — Redis uses `SUBSCRIBE`/`UNSUBSCRIBE` commands on the same connection as regular data. The control channel shares the wire with data. This is a well-understood pattern.
|
|
|
|
3. **Simplicity** — no second schema, no framing differentiation, no prefix bytes. Just a convention on `type` naming.
|
|
|
|
4. **Inspectable** — `__subscribe` messages are as debuggable as any other envelope. They can be logged, traced, and inspected with the same tooling.
|
|
|
|
5. **Extensible** — new control types can be added (`__ping`, `__ack`, etc.) without schema changes.
|
|
|
|
## Decision: Topic-Based Fan-Out (Not Broadcast-All)
|
|
|
|
`dispatchEvent` on fan-out adapters sends only to connections subscribed to the event type, not all connections. This is the correct default because:
|
|
|
|
- Broadcasting all events to all connections wastes bandwidth at O(n) per event
|
|
- Broadcasting leaks data (e.g., a private message sent to all connected clients)
|
|
- The `type:id` pattern already provides the routing key
|
|
- Redis (in our existing adapter) already does topic-level filtering natively
|
|
- uWebSockets.js, Socket.io, and Centrifugo all implement topic/room-based fan-out
|
|
|
|
This requires fan-out adapters to maintain a `subscriptions` map (`Map<string, Set<Connection>>`) from topic string to subscribed connections. The routing key for fan-out is the full topic string (e.g., `"message.sent:conv-123"`), which is the `CustomEvent.type` — the same `type:id` pattern used by `createPubSub`.
|
|
|
|
## Delivery Semantics
|
|
|
|
`__subscribe` and `__unsubscribe` are **fire-and-forget**. They use the same reliable transport as data messages (WebSocket, QUIC stream, etc.). Since these transports guarantee in-order delivery, subscription control messages are either delivered or the connection drops (triggering reconnection and re-subscription). No acknowledgment mechanism is needed.
|
|
|
|
## Consequences
|
|
|
|
- Spokes must declare their interests via `addEventListener` (which triggers `__subscribe` on the wire)
|
|
- Fan-out adapters maintain subscription state (topic → connections map)
|
|
- On disconnection, all subscriptions for that connection are cleaned up
|
|
- On reconnection, the spoke must re-subscribe by calling `addEventListener` again
|
|
- The `__` prefix convention must be documented and reserved — user event types must not start with `__`
|
|
- Direct messaging is a natural consequence: `"direct:${spokeId}"` is just a topic with one subscriber |