docs: triage architecture open questions — amend ADR-006 direction, settle deadline semantics, fix duplicate isResponseEnvelope

This commit is contained in:
2026-05-13 12:14:22 +00:00
parent df3dd82572
commit 5ec6c380a7
9 changed files with 754 additions and 65 deletions

View File

@@ -1,5 +1,5 @@
---
status: stable
status: draft
last_updated: 2026-05-11
---
@@ -57,12 +57,12 @@ const typeboxSchema = FromSchema({
### Purpose
Generates `OperationSpec & { handler }[]` from OpenAPI specs. Each path+method combination becomes an operation with an auto-generated `fetch` handler.
Generates `OperationSpec & { handler }[]` from OpenAPI specs. Each path+method combination becomes an operation with an auto-generated handler. QUERY and MUTATION operations use `OperationHandler` (single-return); SUBSCRIPTION operations use `SubscriptionHandler` (AsyncGenerator).
### `FromOpenAPI(spec, config)`
```ts
function FromOpenAPI(spec: OpenAPISpec, config: HTTPServiceConfig): Array<OperationSpec & { handler: OperationHandler }>
function FromOpenAPI(spec: OpenAPISpec, config: HTTPServiceConfig): Array<OperationSpec & { handler: OperationHandler | SubscriptionHandler }>
```
Processes all paths in the spec. For each path and method combination:
@@ -72,12 +72,21 @@ Processes all paths in the spec. For each path and method combination:
3. **Build output schema** — extracts response schema from `200`/`201` content, falls back to `Type.Unknown()`
4. **Detect operation type**`GET``QUERY`, `text/event-stream` response → `SUBSCRIPTION`, everything else → `MUTATION`
5. **Generate operation id** — uses `operationId` if present, otherwise normalizes `{method}_{path_parts}`
6. **Create handler**auto-generated `fetch` handler that:
- Interpolates path parameters into the URL
- Passes query parameters as search params
- Sends request body as JSON
- Applies auth headers from config
- Returns JSON, text, or `ArrayBuffer` based on response content type
6. **Create handler**based on detected operation type:
- **QUERY / MUTATION**: auto-generated `OperationHandler` (async function) that:
- Interpolates path parameters into the URL
- Passes query parameters as search params
- Sends request body as JSON
- Applies auth headers from config
- Returns JSON, text, or `ArrayBuffer` based on response content type
- Wraps result in `httpEnvelope()` with HTTP metadata
- **SUBSCRIPTION**: auto-generated `SubscriptionHandler` (AsyncGenerator) that:
- Calls `fetch()` with the constructed URL/params
- Reads the response body as a `ReadableStream`
- Parses SSE frames (`data:`, `event:`, `id:` fields per WHATWG specification)
- Yields each parsed event wrapped in `httpEnvelope()` with `contentType: "text/event-stream"`
- Closes the stream on iteration stop or error (in `finally` block)
- Throws `CallError("EXECUTION_ERROR", ...)` on HTTP error status or connection errors
The handler wraps results in `httpEnvelope()` with HTTP metadata (status code, headers, content type). On HTTP error status, it throws `CallError("EXECUTION_ERROR", ...)`. `Value.Cast()` normalization against `outputSchema` is applied by `registry.execute()` and `CallHandler` as part of the shared result pipeline — see [response-envelopes.md](response-envelopes.md#shared-result-pipeline).
@@ -88,7 +97,7 @@ async function FromOpenAPIFile(
path: string,
config: HTTPServiceConfig,
fs?: OpenAPIFS,
): Promise<Array<OperationSpec & { handler: OperationHandler }>>
): Promise<Array<OperationSpec & { handler: OperationHandler | SubscriptionHandler }>>
```
Reads an OpenAPI JSON file. If `fs` is provided, uses `fs.readFile()` (runtime-agnostic). Otherwise, uses Node.js `node:fs/promises`.
@@ -99,7 +108,7 @@ Reads an OpenAPI JSON file. If `fs` is provided, uses `fs.readFile()` (runtime-a
async function FromOpenAPIUrl(
url: string,
config: HTTPServiceConfig,
): Promise<Array<OperationSpec & { handler: OperationHandler }>>
): Promise<Array<OperationSpec & { handler: OperationHandler | SubscriptionHandler }>>
```
Fetches an OpenAPI JSON spec from a URL.
@@ -136,14 +145,94 @@ interface OpenAPIFS {
Injectable filesystem interface for runtime-agnostic file reading. See [ADR-002](decisions/002-fs-injection.md).
### Known Gap: SSE Subscription Handlers
### SSE Subscription Handlers
`FromOpenAPI` correctly detects SSE endpoints (`text/event-stream``SUBSCRIPTION`) but the auto-generated handler does a one-shot `fetch` and returns the response body. For `SUBSCRIPTION` operations, the handler should be an async generator that:
1. Calls `fetch()` with the constructed URL/params
2. Reads the response body as a stream
3. Parses SSE frames (`data:` lines, `event:` lines)
4. Yields each parsed event
5. Cleans up on iteration stop
`FromOpenAPI` detects SSE endpoints by response content type (`text/event-stream``SUBSCRIPTION`) and generates an `AsyncGenerator` handler (not a single-return `OperationHandler`). This makes SSE operations consumable via `subscribe()` and compatible with the call protocol's subscription transport.
#### Handler Pattern
For `SUBSCRIPTION`-type operations, `createHTTPOperation` generates a `SubscriptionHandler` — an `AsyncGenerator` that:
1. Calls `fetch()` with the constructed URL, query params, headers (including auth)
2. Reads the response body as a `ReadableStream`
3. Parses SSE frames from the stream (`data:`, `event:`, `id:` fields per the [SSE specification](https://html.spec.whatwg.org/multipage/server-sent-events.html))
4. Yields each parsed event, wrapped in `httpEnvelope()`
5. Cleans up the stream and response on iteration stop or error
#### SSE Parsing
SSE frames are parsed from the text stream according to the [WHATWG specification](https://html.spec.whatwg.org/multipage/server-sent-events.html):
- **BOM**: A U+FEFF BYTE ORDER MARK at the start of the stream is ignored (per spec §9.2)
- **Line endings**: Lines are split on `\n`, `\r\n`, or `\r` — all three are valid SSE line terminators
- **`data:` lines**: The text after `data:` is appended to the current data buffer. If the text after `data:` starts with a single U+0020 SPACE character, that space is removed (per the spec's "remove U+0020" step). Multiple `data:` lines before a dispatch are joined with `\n`
- **`event:` lines**: Set the event type for the next dispatch (default: `"message"` if no `event:` line)
- **`id:` lines**: Set the last event ID for reconnection
- **`:` lines**: Comments, ignored
- **Empty line**: Dispatches the buffered event (data buffer + event type + last event ID) and resets the parser state
- **Partial lines across `read()` calls**: The parser must retain unprocessed text in a buffer and prepend it to the next chunk. The SSE stream is a continuous text stream; line boundaries don't align with `ReadableStream.read()` boundaries
Parsing function signature:
```ts
function parseSSEFrames(buffer: string): { events: SSEEvent[], remaining: string }
```
Where `SSEEvent` is:
```ts
interface SSEEvent {
data: string // Joined data fields
eventType: string // From "event:" line, default "message"
lastEventId: string // From "id:" line
}
```
The `data` field value is the raw joined string (typically JSON). Consumers decide whether to `JSON.parse()` based on the operation's `outputSchema` or content type heuristics.
**Error handling**: Malformed lines (e.g., lines with no recognized field prefix) are logged as warnings and skipped. The stream continues processing subsequent lines.
Each dispatched event yields:
```ts
yield httpEnvelope(parsedData, {
statusCode: response.status,
headers: responseHeaders,
contentType: "text/event-stream",
})
```
The SSE `event` type and `id` fields are currently **dropped** by the parser — they are not carried in the `ResponseEnvelope`. The `data` field value (the parsed SSE data, typically JSON) is the primary `envelope.data` payload. If consumers need per-event SSE metadata (event type, last event ID), a future `SSEResponseMeta` source type can be added. See [ADR-007](decisions/007-subscription-transport.md) § Open Questions.
#### SSE vs Single-Return Handler
| Aspect | QUERY / MUTATION handler | SUBSCRIPTION handler |
|--------|-------------------------|---------------------|
| Type | `OperationHandler` (returns single value) | `SubscriptionHandler` (AsyncGenerator) |
| Fetch pattern | One request, one response | One request, stream response body |
| Return value | `httpEnvelope(data, meta)` per call | `httpEnvelope(data, meta)` per yield |
| `execute()` | Returns `Promise<ResponseEnvelope>` | Not called via `execute()` — use `subscribe()` |
| Cleanup | Automatic (response consumed) | Must close `ReadableStream` on iteration stop |
#### Error Handling
- **Connection errors** (DNS, timeout): throw `CallError("EXECUTION_ERROR", ...)` from the generator body before the first yield
- **HTTP error status**: throw `CallError("EXECUTION_ERROR", ...)` from the generator body
- **Stream parse errors**: log warning and skip the malformed frame, continue the stream
- **Consumer cancellation**: the generator's `finally` block closes the `ReadableStream` and releases resources
#### Relationship to Call Protocol Subscriptions
SSE operations registered via `FromOpenAPI` are consumable through both:
1. **In-process**: `subscribe(registry, operationId, input, context)` — calls the AsyncGenerator directly, yields `ResponseEnvelope` per SSE event
2. **Remote**: `PendingRequestMap.subscribe(operationId, input, options)` — publishes `call.requested` and yields each `call.responded` event over the configured transport (WebSocket, Redis, etc.)
The handler itself is transport-agnostic. It yields `ResponseEnvelope` values via `httpEnvelope()`. The subscription consumption layer (local `subscribe()` or remote `PendingRequestMap.subscribe()`) handles the routing.
#### Runtime-Agnostic Fetch
The handler uses the global `fetch()` API, which is available in Node.js (18+), Deno, Bun, and modern browsers. No platform-specific `http` module is imported. For runtimes that require a polyfill, the consumer can set `globalThis.fetch` before calling `FromOpenAPI`.
## from_mcp

View File

@@ -1,5 +1,5 @@
---
status: stable
status: draft
last_updated: 2026-05-11
---
@@ -209,11 +209,12 @@ See [call-protocol.md](call-protocol.md) for full semantics.
| Method | Signature | Description |
|--------|-----------|-------------|
| `constructor(eventTarget?)` | `(eventTarget?: EventTarget)` | Creates internal pubsub, wires subscription handlers for responded/error/aborted. |
| `call(operationId, input, options?)` | `Promise<ResponseEnvelope>` | Publish `call.requested`, return Promise that resolves with `ResponseEnvelope` on `call.responded`. |
| `call(operationId, input, options?)` | `Promise<ResponseEnvelope>` | Publish `call.requested`, return Promise that resolves with `ResponseEnvelope` on `call.responded`. For QUERY and MUTATION operations. |
| `subscribe(operationId, input, options?)` | `AsyncIterable<ResponseEnvelope>` | Publish `call.requested`, return AsyncIterable that yields each `call.responded` event. For SUBSCRIPTION operations over remote transport. |
| `respond(requestId, output)` | `void` | Publish `call.responded`. `output` must be `ResponseEnvelope``isResponseEnvelope()` guard throws on raw values. |
| `emitError(requestId, code, message, details?)` | `void` | Publish `call.error`. |
| `abort(requestId)` | `void` | Publish `call.aborted`, reject pending Promise. |
| `getPendingCount()` | `number` | Number of in-flight requests. |
| `abort(requestId)` | `void` | Publish `call.aborted`, reject pending Promise or close repeater. |
| `getPendingCount()` | `number` | Number of in-flight requests (both call and subscribe). |
### `CallHandler`
@@ -260,7 +261,9 @@ async function* subscribe(
Direct subscription execution. Gets the operation, casts its handler to `AsyncGenerator`, yields each value wrapped in `ResponseEnvelope`. If a yielded value is already an envelope (`isResponseEnvelope()`), it passes through. Otherwise, `localEnvelope(value, operationId)` wraps it. Properly cleans up the generator on iteration stop (calls `generator.return()` in `finally`).
This is the synchronous alternative to the call protocol's `call.requested``call.responded` flow for subscriptions. Use `subscribe()` for in-process subscription consumption; use `PendingRequestMap` for cross-transport subscription.
This is the in-process subscription path. For remote subscriptions over a pubsub transport, use `PendingRequestMap.subscribe()` which routes `call.requested` events and yields each `call.responded` envelope.
**SSE operations** (`text/event-stream` endpoints in OpenAPI) are `SUBSCRIPTION`-type operations with `SubscriptionHandler` (AsyncGenerator) handlers. They parse the SSE stream and yield individual events. Both `subscribe()` and `PendingRequestMap.subscribe()` can consume them — `subscribe()` for local, `PendingRequestMap.subscribe()` for remote over WebSocket/Redis transport.
## Env Builder

View File

@@ -1,5 +1,5 @@
---
status: stable
status: draft
last_updated: 2026-05-11
---
@@ -150,9 +150,104 @@ Publishes `call.error`. Used by handlers to send errors.
Looks up the `PendingRequest`, clears its timer, publishes `call.aborted`, rejects the Promise with `CallError(ABORTED, ...)`.
### `subscribe(operationId, input, options?)`
```ts
subscribe(
operationId: string,
input: unknown,
options?: { parentRequestId?: string; deadline?: number; identity?: Identity },
): AsyncIterable<ResponseEnvelope>
```
The subscription counterpart to `call()`. Uses the same event types (`call.requested`, `call.responded`, `call.aborted`, `call.error`) and the same `requestId` correlation, but yields each `call.responded` event instead of resolving after the first one.
1. Generate `requestId` via `crypto.randomUUID()`
2. Create a `Repeater` (from `@alkdev/pubsub`) keyed by `requestId`
3. Publish `call.requested` event with all fields
4. Return the `Repeater` as `AsyncIterable<ResponseEnvelope>`
5. On each `call.responded:{requestId}` event: push `responded.output` to the repeater
6. On `call.error:{requestId}`: push the error (the Repeater throws from the iterator, then closes)
7. On `call.aborted:{requestId}`: close the repeater (iterator completes)
8. On consumer iteration stop: publish `call.aborted` and clean up
This enables remote subscriptions over any `EventTarget` transport. The consumer iterates:
```ts
for await (const envelope of pendingRequestMap.subscribe("opensemaphore.events", { repo: "alkdev/operations" }, { identity })) {
// envelope is a ResponseEnvelope per SSE event
}
```
The handler on the other side must be an `AsyncGenerator` (i.e., a `SubscriptionHandler`). The `CallHandler` on the hub side calls `subscribe()` internally and publishes each yielded envelope as `call.responded`.
### Internal Routing: `call()` vs `subscribe()` per `requestId`
`PendingRequestMap` must track whether a given `requestId` belongs to a `call()` or a `subscribe()` to correctly route `call.responded` events. The internal data structure is:
```ts
type PendingEntry =
| { type: "call"; promise: PendingRequest } // single-resolution Promise
| { type: "subscribe"; repeater: Repeater } // multi-yield AsyncIterable
```
When `call.responded:{requestId}` arrives:
- If `type: "call"` → resolve the promise and delete the entry
- If `type: "subscribe"` → push `responded.output` to the repeater (entry persists until the stream ends)
When `call.error:{requestId}` arrives:
- If `type: "call"` → reject the promise and delete the entry
- If `type: "subscribe"` → push the error to the repeater (Repeater throws), then close the repeater and delete the entry
When `call.aborted:{requestId}` arrives:
- If `type: "call"` → reject the promise and delete the entry
- If `type: "subscribe"` → close the repeater (iterator completes), then delete the entry
Orphaned events (arriving after the consumer has stopped iterating or the entry was already deleted) are silently ignored — no error, no warning, no side effects.
#### Relationship to `subscribe()` direct function
| Aspect | `subscribe()` (direct) | `PendingRequestMap.subscribe()` |
|--------|----------------------|-------------------------------|
| Transport | In-process (calls handler directly) | Remote (via pubsub EventTarget) |
| Handler access | Registry lookup in same process | Hub side: registry lookup, handler call |
| Event routing | None — direct AsyncGenerator | `call.requested` → handler → `call.responded` per yield |
| Use case | Local subscriptions | Cross-process subscriptions (WebSocket, Redis, etc.) |
| Cleanup | Generator `return()` | Publish `call.aborted` + repeater stop |
Both return `AsyncIterable<ResponseEnvelope>` — same consumption pattern, different routing.
### Subscription Lifecycle
#### Error Propagation
| Scenario | `subscribe()` (direct) | `PendingRequestMap.subscribe()` |
|----------|----------------------|-------------------------------|
| Handler throws before first yield | `CallError` propagates to caller | `call.error` published, Repeater throws, consumer's `for await` catches the error |
| Handler throws mid-stream | Error propagates from generator, `finally` block runs | `call.error` published, Repeater throws, consumer's `for await` catches the error |
| Pre-handler errors (ACCESS_DENIED, VALIDATION_ERROR) | `CallError` thrown from `subscribe()` | `CallHandler` catches and publishes `call.error`, Repeater throws to consumer |
| `call.error` event arrives (remote) | N/A (in-process) | Repeater receives error, consumer's `for await` sees it |
#### Cleanup
| Scenario | `subscribe()` (direct) | `PendingRequestMap.subscribe()` |
|----------|----------------------|-------------------------------|
| Consumer stops iteration (`break`) | `generator.return()` called in `finally` | `call.aborted` published, Repeater closed, hub's `CallHandler` receives abort and calls `generator.return()` |
| Consumer's `for await` completes naturally | Generator's `return()` called if `finally` block has cleanup | Repeater stops, no `call.aborted` published |
| Transport disconnect | N/A (in-process) | Hub may continue until timeout/closure; no heartbeat specified yet (see open questions) |
#### Abortion
When a consumer calls `pendingRequestMap.abort(requestId)` for a subscription:
1. `call.aborted:{requestId}` is published to the pubsub
2. The Repeater is closed (consumer's `for await` loop ends)
3. The hub-side `CallHandler` receives `call.aborted`, calls `generator.return()` on the subscription handler (triggers `finally` block for stream cleanup)
When the direct `subscribe()` consumer breaks out of iteration, the generator's `finally` block runs automatically — no `call.aborted` event needed (in-process, no transport).
## CallHandler
`buildCallHandler` creates a function that bridges pubsub events to `OperationRegistry.execute()`. It delegates to `execute()` for the full invocation pipeline (lookup, access control, validation, handler, envelope wrapping, normalization, output validation), taking full ownership of publishing `call.responded`.
`buildCallHandler` creates a function that bridges pubsub events to `OperationRegistry.execute()` or `subscribe()`. It delegates to `execute()` for QUERY/MUTATION operations and to `subscribe()` for SUBSCRIPTION operations.
```ts
function buildCallHandler(config: CallHandlerConfig): CallHandler
@@ -168,12 +263,31 @@ type CallHandler = (event: CallRequestedEvent) => Promise<void>
### Handler Flow
1. Construct `OperationContext` from the event (`requestId`, `parentRequestId`, `identity``trusted` is NOT set, remote calls always run access control)
2. Call `registry.execute(operationId, input, context)` — this performs all validation, access control, and result pipeline
3. On success: publish `call.responded` via `callMap.respond(requestId, envelope)`
4. On failure: `mapError` converts the thrown value to `CallError`, publish `call.error`
2. Look up the operation spec via `registry.getSpec(operationId)`
3. If spec not found → publish `call.error` with `OPERATION_NOT_FOUND`
4. If spec type is `QUERY` or `MUTATION`:
a. Call `registry.execute(operationId, input, context)` — this performs all validation, access control, and result pipeline
b. On success: publish `call.responded` via `callMap.respond(requestId, envelope)`
c. On failure: `mapError` converts the thrown value to `CallError`, publish `call.error`
5. If spec type is `SUBSCRIPTION`:
a. Call `subscribe(registry, operationId, input, context)` — this checks access control and wraps yields in `ResponseEnvelope`
b. For each yielded envelope: publish `call.responded` via `callMap.respond(requestId, envelope)`
c. On generator completion: iterator ends naturally
d. On generator error: `mapError` converts the error, publish `call.error`
e. On `call.aborted:{requestId}`: call `generator.return()` to clean up the subscription
**Note on ADR-006**: [ADR-006](decisions/006-unified-invocation-path.md) specifies that `CallHandler` should be a "thin adapter that calls `registry.execute()`." For QUERY/MUTATION operations, this is the case. For SUBSCRIPTION operations, `CallHandler` must call `subscribe()` directly because `execute()` is designed for single-return operations (it `await`s the handler return value). Making `execute()` aware of SUBSCRIPTION type and routing to `subscribe()` internally would conflate two execution models in one function. The explicit dispatch in `CallHandler` is the correct design — `execute()` handles single-return operations, `subscribe()` handles streaming operations, and `CallHandler` routes between them based on `spec.type`.
**Handling pre-generator errors**: Both `registry.execute()` and `subscribe()` can throw before reaching the handler (e.g., `OPERATION_NOT_FOUND`, `ACCESS_DENIED`, `VALIDATION_ERROR`). `CallHandler` wraps the entire invocation in a try/catch, so these pre-generator errors are caught and published as `call.error`. For subscriptions, this means errors during access control, validation, or spec/handler lookup are reported via `call.error`, not silently swallowed.
**Key change**: In the pre-envelope model, handlers were responsible for publishing `call.responded` themselves (the handler return value was discarded). In the envelope model, `CallHandler` owns wrapping and publishing. Handler return values are captured and wrapped. This ensures every response goes through the envelope pipeline — no raw values can bypass it.
### Subscription Handling Detail
For `SUBSCRIPTION` operations, the `CallHandler` iterates the async generator and publishes each yield as `call.responded`. This means a single `call.requested` event can produce **multiple** `call.responded` events with the same `requestId`. The `PendingRequestMap.subscribe()` method consumes this stream.
The `call.aborted` event terminates the subscription. When the hub-side `CallHandler` receives `call.aborted:{requestId}`, it calls `generator.return()` on the active subscription handler. This triggers the generator's `finally` block (stream cleanup, resource release).
### MCP and OpenAPI Handlers
Adapter handlers (from `from_mcp` and `from_openapi`) return pre-built `ResponseEnvelope` instances via `mcpEnvelope()` and `httpEnvelope()` factory functions. When `CallHandler` detects `isResponseEnvelope()` on the result, it passes through without re-wrapping. This means adapter metadata (HTTP status codes, MCP `isError` flags) is preserved.
@@ -272,9 +386,33 @@ The call protocol is transport-agnostic. The `PubSub` event target determines ho
|-----------|----------|-----------------|
| In-process | Local hub operations | Browser `EventTarget` (default) |
| Redis | Cross-process events | `RedisEventTarget` (from `@alkdev/pubsub`) |
| WebSocket | Hub ↔ spoke bidirectional | `WebSocketEventTarget` (future) |
| WebSocket client | Spoke → Hub bidirectional | `WebSocketClientEventTarget` (from `@alkdev/pubsub`) |
| WebSocket server | Hub → Spoke fan-out | `WebSocketServerEventTarget` (from `@alkdev/pubsub`) |
Same protocol, same event shapes, same `PendingRequestMap` — different `eventTarget`.
Same protocol, same event shapes, same `PendingRequestMap` — different `eventTarget`. Both `call()` and `subscribe()` work over any transport — the only difference is consumption pattern (single-resolution Promise vs. AsyncIterable).
### WebSocket Topology
The WebSocket event targets from `@alkdev/pubsub` carry call protocol events between hub and spoke processes:
```
Spoke process Hub process
│ │
│ createPubSub({ │ createPubSub({
│ eventTarget: │ eventTarget:
│ WebSocketClientET │ WebSocketServerET
│ }) │ })
│ │
│ pendingRequestMap │
│ .call(op, input) ── ws ──────────────> │ CallHandler → execute()
│ .subscribe(op, input) ── ws ─────────> │ CallHandler → subscribe()
│ │
│ <── call.responded ────────────────── ws │
│ <── call.responded ────────────────── ws │ (multiple for subscribe)
│ <── call.error ────────────────────── ws │
```
The WebSocket client adapter handles `__subscribe`/`__unsubscribe` control events automatically — when `PendingRequestMap` subscribes to `call.responded:{requestId}`, the client adapter sends `__subscribe` for that topic, and the server adapter routes only matching events to that spoke. See [ADR-003](../../../@alkdev/pubsub/docs/architecture/decisions/003-subscription-control-protocol.md) in `@alkdev/pubsub`.
## Subscribe (Direct)

View File

@@ -1,6 +1,6 @@
---
status: accepted
last_updated: 2026-05-11
last_updated: 2026-05-13
---
# ADR-006: Unified Invocation Path
@@ -35,9 +35,9 @@ This ADR depends on ADR-005 (response envelopes) being implemented in source fir
## Decision
**Unify on `execute()` as the single invocation entry point.** All consumers — local in-process code, `buildEnv()`, and future worker pool routers — call `registry.execute()` and get the same behavior: envelope wrapping, result pipeline, access control, consistent error handling.
**Unify on `execute()` as the single invocation entry point.** All consumers — local in-process code, `buildEnv()`, and future spoke/hub routers — call `registry.execute()` and get the same behavior: envelope wrapping, result pipeline, access control, consistent error handling.
The call protocol (`call.requested` / `call.responded` / `CallHandler` / `PendingRequestMap`) becomes an **internal transport mechanism** for routing invocations across process boundaries. It is not a public invocation path — it's the plumbing that `execute()` uses when the target handler is in another process.
The call protocol (`call.requested` / `call.responded` / `CallHandler` / `PendingRequestMap`) is the **primary integration surface** for bi-directional cross-process communication. Spokes and hubs exchange call protocol events over pubsub transports (WebSocket, Redis). `PendingRequestMap`, `CallHandler`, and `CallEventSchema` remain public exports — they are the API that spoke and hub SDKs integrate against. This is not internal plumbing; it is the product's interop layer.
### Architectural model
@@ -121,7 +121,7 @@ The `customAuth` field on `AccessControl` is declared but not yet enforced anywh
2. **`buildEnv()` always uses `registry.execute()`.** The `callMap` option is removed from `buildEnv()`. `OperationEnv` functions call `execute()` directly. Nested calls propagate the same `context` (plus `trusted: true`).
3. **Call protocol is internal transport.** `PendingRequestMap`, `CallHandler`, and `CallEventSchema` become internal implementation details, not part of the public invocation API. They exist for cross-process routing only.
3. **Call protocol is the integration surface.** `PendingRequestMap`, `CallHandler`, and `CallEventSchema` remain public exports. They are the API that spoke and hub SDKs integrate against for cross-process communication. Consumers construct `PendingRequestMap` with a transport `EventTarget` and use `call()` / `subscribe()` directly. `buildCallHandler()` bridges incoming events to `registry.execute()`.
4. **`CallHandler` calls `registry.execute()` on the worker side.** Instead of duplicating lookup, validation, and access control, the worker-side `CallHandler` becomes a thin adapter:
```ts
@@ -148,9 +148,9 @@ The `customAuth` field on `AccessControl` is declared but not yet enforced anywh
### What stays the same
- **`PendingRequestMap`**: Still needed for routing `call.responded` back to the correct promise. Internal transport plumbing.
- **`CallEventSchema`**: Still the wire format for cross-process communication. Internal.
- **`subscribe()`**: Direct in-process subscriptions remain. They call the handler's async generator directly with envelope wrapping. `subscribe()` also applies access control when `identity` is present (consistent with `execute()`). Envelope wrapping per yield is addressed by ADR-005.
- **`PendingRequestMap`**: Routes `call.responded` back to the correct promise or Repeater. The primary API for spoke/hub callers to invoke remote operations over a pubsub transport.
- **`CallEventSchema`**: The wire format for cross-process communication. The interop contract between hub and spoke.
- **`subscribe()`**: Direct in-process subscriptions remain. They call the handler's async generator directly with envelope wrapping. `subscribe()` also applies access control when `identity` is present (consistent with `execute()`). Envelope wrapping per yield is addressed by ADR-005. For remote subscriptions over a transport, use `PendingRequestMap.subscribe()` (see ADR-007).
- **Adapters (`from_mcp`, `from_openapi`)**: Register handlers in the registry as before. Their handlers return `ResponseEnvelope` instances (via factory functions) as the envelope spec describes.
### What changes
@@ -162,7 +162,7 @@ The `customAuth` field on `AccessControl` is declared but not yet enforced anywh
| `execute()` doesn't wrap in envelope | `execute()` applies result pipeline (detect → wrap → normalize → validate) |
| `buildEnv()` toggles `execute()` vs `callMap.call()` | `buildEnv()` always uses `execute()` |
| `CallHandler` duplicates handler invocation | `CallHandler` calls `registry.execute()` internally |
| `callMap` is a public concept | Call protocol is internal transport plumbing |
| `callMap` is a public concept | Call protocol is the public cross-process integration API |
| Two different invocation guarantees | Same behavior regardless of local/remote |
| `OperationContext` has no `trusted` field | `OperationContext` gains `trusted?: boolean` |
| Identity not propagated through `buildEnv()` | `buildEnv()` propagates identity and sets `trusted: true` |
@@ -202,7 +202,7 @@ Subscriptions are excluded from `OperationEnv` (as currently) — `buildEnv()` o
### Negative
- **Performance for local calls**: `execute()` now applies access control, envelope wrapping, `Value.Cast()` normalization, and output validation on every call, even trusted same-process calls. The `trusted` flag skips redundant scope checks, but envelope wrapping and validation remain. Estimated overhead: ~1-5μs per call for envelope construction + detection + access check. This is acceptable for our use case (operations are typically milliseconds to seconds). Benchmark before stabilizing.
- **API change**: Removing `callMap` from `buildEnv()` and making call protocol internal is a breaking change. Package is pre-1.0; consumers are coordinated.
- **API change**: Removing `callMap` from `buildEnv()` is a breaking change. `buildCallHandler()` now requires `callMap` explicitly rather than the `CallHandler` owning transport configuration. Package is pre-1.0; consumers are coordinated.
- **Complexity in `execute()`**: Routing logic (local vs. remote) adds conditional paths inside `execute()`. This is simpler than the current external toggle, but `execute()` becomes more complex internally.
### Risks
@@ -217,11 +217,11 @@ Subscriptions are excluded from `OperationEnv` (as currently) — `buildEnv()` o
2. **Update `execute()`** — return `Promise<ResponseEnvelope<TOutput>>`, apply result pipeline (detect → wrap → normalize → validate), add access control check.
3. **Add `trusted` to `OperationContext`** — internal-only, set by `buildEnv()`.
4. **Update `buildEnv()`** — remove `callMap` option, always call `execute()`, propagate `context` with `trusted: true`.
5. **Simplify `CallHandler`** — thin adapter that calls `registry.execute()`, catches errors, publishes events.
5. **Simplify `CallHandler`** — thin adapter that calls `registry.execute()`, catches errors, publishes events. Now requires explicit `callMap` parameter.
6. **Update `subscribe()`** — add access control check, wrap yields in `ResponseEnvelope`.
7. **Update `OperationEnv` return type** — `Promise<unknown>` → `Promise<ResponseEnvelope>`.
8. **Add remote routing to `execute()`** — when EventTarget transport is configured on registry and handler is not local, publish `call.requested` and await response. (Deferred until worker pool is built.)
9. **Move call protocol exports** — `PendingRequestMap`, `CallHandler`, `CallEventSchema` move from public barrel to internal. `call()` and `respond()` become internal APIs.
8. **Add remote routing to `execute()`** — when EventTarget transport is configured on registry and handler is not local, publish `call.requested` and await response. (Deferred until spoke/hub transport is built.)
9. ~~**Move call protocol exports**~~**Struck.** Call protocol types remain public exports as the integration surface for spoke/hub SDKs.
## References

View File

@@ -0,0 +1,199 @@
---
status: draft
last_updated: 2026-05-13
---
# ADR-007: Subscription Transport for SSE and Remote Streaming
## Context
`FromOpenAPI` detects SSE endpoints (`text/event-stream``SUBSCRIPTION`) but the current handler does a one-shot `fetch` and returns the response body. This means:
1. **Handler type mismatch**: `SUBSCRIPTION` operations get an `OperationHandler` (single-return) instead of a `SubscriptionHandler` (AsyncGenerator). They can't be consumed via `subscribe()`.
2. **No SSE stream parsing**: The handler returns the raw response body instead of yielding individual SSE events.
3. **No remote subscription transport**: `PendingRequestMap.call()` resolves after one `call.responded` event. There's no way to consume a subscription over a remote transport (WebSocket, Redis, etc.) — you get one response and the promise resolves, even if the operation yields multiple events.
4. **PubSub WebSocket is underutilized**: `@alkdev/pubsub` provides WebSocket client and server event targets with bidirectional `__subscribe`/`__unsubscribe` control events. The operations call protocol (`call.requested`/`call.responded`) could ride on this transport for remote subscriptions, but the plumbing doesn't exist yet.
### The core insight: `call ≡ subscribe`
The call protocol already defines this equivalence at the protocol level — same event types, same `requestId` correlation, same `PendingRequestMap`. The difference is consumption pattern:
- **`call`**: Publish `call.requested`, resolve on first `call.responded``Promise<ResponseEnvelope>`
- **`subscribe`**: Publish `call.requested`, yield each `call.responded``AsyncIterable<ResponseEnvelope>`
The `PendingRequestMap` currently only implements the `call` pattern. The `subscribe` pattern is missing from the remote transport path.
### PubSub transport
`@alkdev/pubsub` provides `createPubSub({ eventTarget })` where the event target can be:
| Transport | Use Case | EventTarget |
|-----------|----------|-------------|
| In-process | Local operations | Browser `EventTarget` (default) |
| Redis | Cross-process events | `RedisEventTarget` |
| WebSocket client | Spoke → Hub | `WebSocketClientEventTarget` |
| WebSocket server | Hub → Spoke fan-out | `WebSocketServerEventTarget` |
When `PendingRequestMap` is constructed with a WebSocket event target, all `call.requested`/`call.responded`/`call.error`/`call.aborted` events flow over the WebSocket. This already works for single-request/response calls. For subscriptions, we need the same events to flow continuously until the subscription is stopped.
## Decision
### 1. SSE subscription handlers are AsyncGenerators
`FromOpenAPI` generates `SubscriptionHandler` (AsyncGenerator) for `SUBSCRIPTION`-type operations. The handler:
1. Calls `fetch()` with the constructed URL, params, and auth headers
2. Reads the response body as a `ReadableStream`
3. Parses SSE frames per the WHATWG specification (`data:`, `event:`, `id:` fields)
4. Yields each parsed event wrapped in `httpEnvelope()` with `contentType: "text/event-stream"`
5. Closes the stream on iteration stop (in `finally` block)
This makes SSE operations consumable via `subscribe()` and compatible with the call protocol's subscription transport.
### 2. `PendingRequestMap.subscribe()` for remote subscriptions
Add a `subscribe()` method to `PendingRequestMap` that returns `AsyncIterable<ResponseEnvelope>`:
```ts
subscribe(
operationId: string,
input: unknown,
options?: { parentRequestId?: string; deadline?: number; identity?: Identity },
): AsyncIterable<ResponseEnvelope>
```
- Publishes `call.requested` (same as `call()`)
- Uses a `Repeater` (from `@alkdev/pubsub`) keyed by `requestId`
- Pushes each `call.responded:{requestId}` event's `output` field to the repeater
- On `call.error:{requestId}`: throws from the iterator
- On `call.aborted:{requestId}`: closes the iterator (completes)
- On consumer iteration stop: publishes `call.aborted` and cleans up
The consumer iterates identically to `subscribe()`:
```ts
for await (const envelope of pendingRequestMap.subscribe("opensemaphore.events", input, { identity })) {
// process each SSE event envelope
}
```
### 3. `CallHandler` dispatches on operation type
`CallHandler` checks the operation type and routes accordingly:
- **QUERY / MUTATION**: Call `registry.execute()`, publish single `call.responded` or `call.error`
- **SUBSCRIPTION**: Call `subscribe()`, publish `call.responded` for each yield, handle `call.aborted` by calling `generator.return()`
This means a single `call.requested` can produce multiple `call.responded` events with the same `requestId` — one per yielded value from the subscription handler.
### 4. SSE events use existing `httpEnvelope`
Individual SSE events are wrapped in `httpEnvelope()` with `contentType: "text/event-stream"`. No new `ResponseSource` type is needed because:
- SSE events are HTTP responses — the initial connection establishes status and headers
- The `contentType` field in `HTTPResponseMeta` already distinguishes `"text/event-stream"` from `"application/json"`
- Each yielded event carries the same `statusCode` and `headers` from the initial response
- Adding a separate `"sse"` source type would be premature — the `http` source with `contentType` discrimination is sufficient
If consumers need per-event SSE metadata (event type, last event ID), this can be carried in `envelope._meta` on the `OperationSpec` or added as a future `SSEResponseMeta` type.
### 5. SSE handler is runtime-agnostic
The handler uses the global `fetch()` API (available in Node 18+, Deno, Bun, browsers). `ReadableStream` and `TextDecoderStream` are also web standards. No platform-specific imports.
For environments where `fetch()` is not available, the consumer must provide a polyfill before calling `FromOpenAPI`. We do not inject or configure `fetch` — it is a global.
## Event Flow: Remote SSE Subscription
```
Spoke (client) Hub (server)
│ │
│─── call.requested ───────────────────────> │
│ {requestId, operationId, │
│ input, identity} │
│ │─ CallHandler checks type=SUBSCRIPTION
│ │─ subscribe(registry, opId, input, ctx)
│ │─ SSE handler: fetch() → parse stream
│ │
│<── call.responded:{requestId} ────────── │ (yield #1)
│ {output: ResponseEnvelope} │
│ │
│<── call.responded:{requestId} ────────── │ (yield #2)
│ {output: ResponseEnvelope} │
│ │
│ ... │
│ │
│─── call.aborted ────────────────────────> │ (consumer done, stops iteration)
│ {requestId} │─ generator.return() closes SSE stream
```
The spoke uses `PendingRequestMap.subscribe()` with a WebSocket event target. Each `call.responded` event flows through the pubsub transport. The hub-side `CallHandler` iterates the subscription handler's AsyncGenerator and publishes each yield.
## What Stays the Same
- **`subscribe()` direct function**: Local in-process subscription consumption. Calls the handler's AsyncGenerator directly, wraps yields in `ResponseEnvelope`. No pubsub transport involved.
- **`PendingRequestMap.call()`**: Single-request/response pattern. Publishes `call.requested`, resolves on first `call.responded`. Used for QUERY and MUTATION operations over remote transport.
- **`OperationHandler` vs `SubscriptionHandler` types**: `OperationHandler` returns a single value; `SubscriptionHandler` is an AsyncGenerator. The registry stores both.
- **Response envelope model**: All values are wrapped in `ResponseEnvelope` at the protocol boundary, regardless of transport.
- **`httpEnvelope()` factory**: Used by OpenAPI adapter for both single-return and SSE handlers.
## What Changes
| Before | After |
|--------|-------|
| SSE operations get `OperationHandler` (single-return) | SSE operations get `SubscriptionHandler` (AsyncGenerator) |
| SSE handler does one-shot `fetch` | SSE handler streams response body, yields per SSE event |
| `PendingRequestMap` has `call()` only | `PendingRequestMap` gains `subscribe()` method |
| `CallHandler` always calls `registry.execute()` | `CallHandler` dispatches: `execute()` for QUERY/MUTATION, `subscribe()` for SUBSCRIPTION |
| No remote subscription transport | Remote subscriptions over pubsub (WebSocket, Redis, etc.) |
## Consequences
### Positive
- **SSE operations work as subscriptions**: Consumable via `subscribe()` and `PendingRequestMap.subscribe()`
- **Unified subscription model**: Same `call ≡ subscribe` semantics for local and remote
- **Leverages existing pubsub transport**: No new transport code needed — `PendingRequestMap.subscribe()` uses the same `EventTarget` as `PendingRequestMap.call()`
- **Consistent envelope model**: SSE events wrapped in `httpEnvelope()`, same as single-return HTTP responses
- **Transport-agnostic**: SSE handler works with any pubsub transport in `PendingRequestMap`
### Negative
- **`PendingRequestMap` complexity**: Managing both `call()` and `subscribe()` subscriptions in the same instance introduces complexity for lifecycle management. The `call.responded:{requestId}` topic now either resolves a promise or pushes to a repeater, depending on whether the request was initiated via `call()` or `subscribe()`.
- **Backpressure on SSE streams**: If the hub produces SSE events faster than the spoke consumes them, events buffer in the WebSocket/transport. The `maxBufferedAmount` backpressure policy on the server-side event target disconnects slow consumers, but this is abrupt. A more graceful flow-control mechanism is deferred.
- **Subscription lifecycle**: `call.aborted` must reach the hub-side handler to close the SSE stream. If the transport drops, the hub may continue the SSE stream until it times out or the TCP connection closes. This is acceptable for SSE (which keeps the HTTP connection open), but requires consideration for long-lived subscriptions.
### Risks
- **Multiple `call.responded` per `requestId`**: The current `PendingRequestMap.call()` assumes one `call.responded` per `requestId` and then removes the pending request. With `subscribe()`, the `requestId` maps to a Repeater, not a promise. `PendingRequestMap` must track whether a `requestId` is a `call` or `subscribe` to route correctly. This increases internal complexity. See call-protocol.md § Internal Routing for the data structure.
- **SSIS parsing robustness**: The SSE specification has edge cases (reconnection via `Last-Event-ID`, BOM handling, empty data fields, partial lines across `read()` calls). The implementation must handle these gracefully but a comprehensive SSE parser test suite is needed.
- **Generator error propagation**: If the `SubscriptionHandler` throws mid-stream, `CallHandler` must catch the error, publish `call.error`, and clean up. This is straightforward but must be tested. Pre-generator errors (ACCESS_DENIED, VALIDATION_ERROR) are also caught and published as `call.error`.
- **Backpressure on SSE streams**: If the hub produces SSE events faster than the spoke consumes them, events buffer in the WebSocket/transport. The `maxBufferedAmount` backpressure policy on the server-side event target disconnects slow consumers, but this is abrupt. A more graceful flow-control mechanism is deferred.
## Open Questions
1. **SSEResponseMeta**: SSE events currently use `httpEnvelope()` with `contentType: "text/event-stream"`. The SSE `event` type and `id` fields are **dropped** by the current parser — they are not carried in the `ResponseEnvelope`. This means consumers that need per-event SSE metadata (event type for dispatch, last event ID for reconnection) cannot access it. The `data` field value (typically JSON) is the primary `envelope.data` payload. A future `SSEResponseMeta` with `source: "sse"`, `eventType: string`, `lastEventId: string` could carry this metadata. Deferred until usage patterns confirm the need.
2. **Deadline for subscriptions** — Resolved: `deadline` always means "idle timeout" (max time between yielded values), not a hard wall-clock cutoff. This accommodates two subscription patterns with a single field:
- **Streaming** (LLM token streams, SSE event feeds): If no envelope arrives within `deadline` ms, the subscription is considered dead. This is the natural timeout — the stream is expected to produce events continuously.
- **Watchdog** (network failure listeners, resource monitors): These subscriptions must stay alive indefinitely and fire only on rare events. A `deadline` alone would kill them. The handler must yield a heartbeat envelope at an interval shorter than `deadline` to prove liveness. The heartbeat is simply a `ResponseEnvelope` with a distinguishable shape (e.g., `_meta: { heartbeat: true }`); the consumer can filter or ignore heartbeats without protocol changes.
No new protocol fields are needed. `deadline` semantics are uniform: "if I don't hear from you for this long, you're dead." Streaming handlers naturally satisfy this by yielding data. Watchdog handlers satisfy it by explicitly yielding no-op heartbeats. Hard wall-clock cutoffs are a consumer-side concern (call `abort()` when needed).
3. **Reconnection**: The SSE specification includes `Last-Event-ID` for automatic reconnection. Should the SSE handler support automatic reconnection with `Last-Event-ID`, or should that be the consumer's responsibility? A reconnecting handler would need to re-fetch with the `Last-Event-ID` header and resume the generator. This is complex and deferred until usage patterns confirm the need.
## References
- [adapters.md](../adapters.md) — FromOpenAPI adapter internals
- [call-protocol.md](../call-protocol.md) — Call protocol spec, PendingRequestMap, CallHandler
- [response-envelopes.md](../response-envelopes.md) — Envelope types and factory functions
- [ADR-006](006-unified-invocation-path.md) — Unified invocation path (execute as single entry point)
- SSE specification: https://html.spec.whatwg.org/multipage/server-sent-events.html
- `@alkdev/pubsub` event targets: WebSocket client, WebSocket server, Redis

View File

@@ -325,7 +325,9 @@ Key differences from current behavior:
### OpenAPI Adapter (`from_openapi.ts`)
Handler behavior change:
#### QUERY / MUTATION handler
Handler behavior for single-return operations:
```ts
handler: async (input, context) => {
@@ -358,6 +360,55 @@ handler: async (input, context) => {
- Success responses → wrapped in `httpEnvelope` with full HTTP metadata
- Headers are a `Record<string, string>` snapshot (multi-value headers joined with `, ` per fetch spec)
#### SUBSCRIPTION handler (SSE)
For `SUBSCRIPTION`-type operations (detected by `text/event-stream` in response content type), the handler is an `AsyncGenerator` that:
```ts
handler: async function* (input, context) => {
// ... URL construction and fetch logic (same as QUERY/MUTATION) ...
const response = await fetch(url, fetchOptions)
if (!response.ok) {
throw new CallError("EXECUTION_ERROR", `HTTP ${response.status}: ${response.statusText}`)
}
// Parse the SSE stream
const reader = response.body!.getReader()
const decoder = new TextDecoder()
let buffer = ""
let eventType = ""
let data = ""
let lastEventId = ""
try {
while (true) {
const { done, value } = await reader.read()
if (done) break
buffer += decoder.decode(value, { stream: true })
// Parse SSE frames from buffer
// ... (see adapters.md for full parsing rules) ...
// When a complete event is dispatched:
yield httpEnvelope(parsedData, {
statusCode: response.status,
headers: Object.fromEntries(response.headers.entries()),
contentType: "text/event-stream",
})
}
} finally {
reader.releaseLock()
}
}
```
- Each SSE event is yielded as a `ResponseEnvelope` with `meta.contentType: "text/event-stream"`
- The SSE `event` type and `id` fields are not carried in the envelope — a future `SSEResponseMeta` source type may be added if per-event metadata is needed
- On HTTP error status → throw `CallError` from the generator body before first yield
- On stream parse error → log warning, skip malformed frame, continue
- `finally` block closes the `ReadableStream` reader
### Value.Cast() for Data Normalization
The shared result pipeline (defined above) includes `Value.Cast()` normalization as step 3. This section provides additional context on how `Value.Cast()` works and why it matters for each source:
@@ -379,7 +430,17 @@ Operations with `Type.Unknown()` `outputSchema` (typically MCP tools without `ou
})
```
## `outputSchema` and Validation
## Type Erasure at Runtime Boundaries
`ResponseEnvelope<T>` carries a compile-time type parameter `T`, but at the call protocol boundary (pubsub serialization, WebSocket transport), `T` is erased to `unknown`. This means:
- `registry.execute<TInput, TOutput>()` returns `Promise<ResponseEnvelope<TOutput>>` — type-safe at compile time
- `PendingRequestMap.call()` returns `Promise<ResponseEnvelope>``TOutput` is not available (the caller doesn't know the operation's output type without a spec lookup)
- `subscribe()` yields `AsyncGenerator<ResponseEnvelope, void, unknown>` — similarly untyped
The runtime guarantee for `envelope.data` shape is `outputSchema` + `Value.Cast()` normalization (step 3 of the shared result pipeline). When `outputSchema` is `Type.Unknown()`, no runtime shape guarantee exists — the consumer must handle arbitrary data.
## `outputSchema` and Validation`
`outputSchema` on `OperationSpec` validates `envelope.data`, not the full `ResponseEnvelope`. The envelope has its own `ResponseEnvelopeSchema` for call protocol validation. This keeps `outputSchema` focused on business data shape — no envelope schema bloat, and existing `outputSchema` definitions don't need changes.
@@ -405,9 +466,9 @@ The normalization step is optional: if `outputSchema` is `Type.Unknown()`, norma
1. **Client abstraction** — Envelopes provide metadata and `unwrap()` provides simple access, but real usage may reveal patterns that justify a higher-level abstraction: e.g., a client that auto-handles MCP `isError` results (throwing on error content vs. returning it), or one that extracts pagination cursors from HTTP headers. Deferred until usage patterns from alkhub and opencode confirm whether `unwrap()` + `meta.source` dispatch is sufficient or whether a typed "client" per source adds real value.
2. **Subscription envelopes**`subscribe()` wraps each yield in `ResponseEnvelope`. For long-running subscriptions, `localResponseMeta.timestamp` updates per yield. Whether subscriptions need additional metadata (e.g., sequence numbers, cursor positions) is an open question for future iteration.
2. **SSEResponseMeta** — SSE events currently use `httpEnvelope()` with `contentType: "text/event-stream"`. The SSE `event` type and `id` fields are **dropped** by the parser — they are not available in the `ResponseEnvelope`. The `data` field value (typically JSON) is the primary `envelope.data` payload. A future `SSEResponseMeta` with `source: "sse"`, `eventType: string`, `lastEventId: string` could carry this per-event metadata if usage patterns confirm the need. See [ADR-007](decisions/007-subscription-transport.md).
3. **`respond()` visibility** — Currently public on `PendingRequestMap`. After CallHandler takes ownership of publishing, `respond()` may become internal-only to enforce the envelope invariant.
3. **`respond()` visibility** — Resolved: `respond()` remains public on `PendingRequestMap`. The call protocol is the integration surface for spoke/hub SDKs (see amended ADR-006), which means spokes need `respond()` for publishing `call.responded` events back to the hub. The envelope invariant is still enforced by the `isResponseEnvelope()` guard.
4. **Envelope directionality (serving)** — The current design covers **consuming** remote operations (MCP, OpenAPI) and wrapping their results. The reverse direction — **serving** local operations via MCP or OpenAPI — is not yet addressed. When exposing operations as an MCP server, results must be wrapped in MCP's `CallToolResult` format (`structuredContent` + `content`). When exposing as OpenAPI, results must be serialized as HTTP response bodies. How envelopes interact with this serving direction is an open design question.
@@ -435,20 +496,28 @@ The following documentation changes have been completed:
| `adapters.md` | `from_mcp` | `outputSchema` extracted when available, via `FromSchema`. Falls back to `Type.Unknown()`. | ✅ |
| `adapters.md` | `from_openapi` | Handler returns `httpEnvelope()`. Error on HTTP error status still throws `CallError`. | ✅ |
The following **code** changes have been completed:
The following **documentation** changes are pending for the subscription transport feature:
| Document | Section | Change | Status |
|----------|---------|--------|--------|
| `adapters.md` | `FromOpenAPI` SSE handlers | Subscription handler as AsyncGenerator, SSE parsing, per-yield envelope | ✅ (doc updated) |
| `call-protocol.md` | PendingRequestMap | Add `subscribe()` method for remote subscriptions | ✅ (doc updated) |
| `call-protocol.md` | CallHandler | Dispatch on operation type: `execute()` for QUERY/MUTATION, `subscribe()` for SUBSCRIPTION | ✅ (doc updated) |
| `call-protocol.md` | Transport Mapping | Add WebSocket topology diagram, mention `subscribe()` over transport | ✅ (doc updated) |
| `api-surface.md` | PendingRequestMap | Add `subscribe()` method to the table | ✅ (doc updated) |
| `api-surface.md` | Subscribe | Add SSE operations note, mention `PendingRequestMap.subscribe()` for remote | ✅ (doc updated) |
| `decisions/007-subscription-transport.md` | New ADR | SSE subscription handler, PendingRequestMap.subscribe(), CallHandler dispatch | ✅ (doc created) |
| `response-envelopes.md` | OpenAPI adapter | Separate QUERY/MUTATION handler from SUBSCRIPTION handler (SSE) | ✅ (doc updated) |
| `response-envelopes.md` | Open Questions | Replace subscription envelopes question with SSEResponseMeta question | ✅ (doc updated) |
The following **code** changes are pending:
| Code | Change | Status |
|------|--------|--------|
| `src/response-envelope.ts` | New file: types, factory functions, detection, schemas | ✅ |
| `src/registry.ts` | `execute()` returns `Promise<ResponseEnvelope<TOutput>>` | ✅ |
| `src/call.ts` | `CallHandler` captures return value, wraps in envelope, publishes `call.responded` | ✅ |
| `src/call.ts` | `CallEventSchema` `output` field changes to `ResponseEnvelopeSchema` | ✅ |
| `src/call.ts` | `PendingRequestMap.respond()` adds `isResponseEnvelope()` guard | ✅ |
| `src/call.ts` | `PendingRequestMap.call()` resolves with `ResponseEnvelope` | ✅ |
| `src/subscribe.ts` | `subscribe()` wraps yields in `ResponseEnvelope` | ✅ |
| `src/env.ts` | `buildEnv()` functions return `Promise<ResponseEnvelope>` | ✅ |
| `src/from_mcp.ts` | Handler returns `mcpEnvelope()`, extracts `outputSchema`, uses `structuredContent` | ✅ |
| `src/from_openapi.ts` | Handler returns `httpEnvelope()` | ✅ |
| `src/from_openapi.ts` | Generate `SubscriptionHandler` (AsyncGenerator) for SUBSCRIPTION operations, parse SSE stream, yield per-event | ❌ Not started |
| `src/call.ts` | Add `PendingRequestMap.subscribe()` method using Repeater from `@alkdev/pubsub` | ❌ Not started |
| `src/call.ts` | Update `CallHandler` to dispatch on operation type | ❌ Not started |
| `src/subscribe.ts` | Ensure `subscribe()` handles `httpEnvelope` detection for SSE yields | ✅ Already handles envelopes |
## References

View File

@@ -2,7 +2,7 @@ import { Type, type Static } from "@alkdev/typebox";
import { createPubSub, type PubSub } from "@alkdev/pubsub";
import { OperationRegistry } from "./registry.js";
import { CallError, InfrastructureErrorCode, mapError } from "./error.js";
import { ResponseEnvelopeSchema } from "./response-envelope.js";
import { ResponseEnvelopeSchema, isResponseEnvelope } from "./response-envelope.js";
import type { ResponseEnvelope } from "./response-envelope.js";
import type { Identity, OperationContext, AccessControl } from "./types.js";
@@ -229,10 +229,3 @@ export function checkAccess(accessControl: AccessControl, identity: Identity): b
return true;
}
function isResponseEnvelope(value: unknown): value is ResponseEnvelope {
if (typeof value !== "object" || value === null) return false;
const obj = value as Record<string, unknown>;
if (!("data" in obj) || !("meta" in obj)) return false;
if (typeof obj.meta !== "object" || obj.meta === null) return false;
return ["local", "http", "mcp"].includes((obj.meta as Record<string, unknown>).source as string);
}

View File

@@ -0,0 +1,40 @@
# Task: Envelope Directionality — Serving Operations via MCP and HTTP
**Priority**: Medium (blocks hub serving operations to spokes)
**Dependencies**: ADR-005 (response envelopes implemented), ADR-006 (unified invocation path)
**Architecture docs**: [response-envelopes.md](../docs/architecture/response-envelopes.md) § Open Questions #4
## Problem
The current design covers **consuming** remote operations (MCP, OpenAPI) and wrapping their results in `ResponseEnvelope`. The reverse direction — **serving** local operations via MCP or OpenAPI — is not yet addressed.
When the hub exposes operations as an MCP server, results must be wrapped in MCP's `CallToolResult` format (`structuredContent` + `content`). When exposing as OpenAPI, results must be serialized as HTTP response bodies. How `ResponseEnvelope` results get converted to these protocol-specific formats is an open design question.
## Design Questions to Answer
1. **Where does the conversion happen?** Does the hub call `registry.execute()`, get a `ResponseEnvelope`, and then convert it? Or does a serving adapter register as a handler that wraps and returns protocol-native types?
2. **What happens to envelope metadata when serving?** A local operation's `ResponseEnvelope` has `meta: { source: "local", operationId, timestamp }`. When served as MCP, should this become `_meta` on the `CallToolResult`? When served as HTTP, should the timestamp become a `X-Operation-Timestamp` header?
3. **Access control direction for serving**: When serving operations, access control is "what operations does this connected spoke have access to?" vs the consuming direction which is "does the caller have scopes for this operation?"
4. **How do subscription (SUBSCRIPTION) operations work in the serving direction?** When a spoke calls an MCP tool that maps to a SUBSCRIPTION operation on the hub, the result is a stream. How does this map to MCP's response model? MCP doesn't natively have streaming tool results.
5. **Relationship to existing adapters**: `from_mcp` and `from_openapi` are currently one-directional (consume external → register operations). Would "to" variants (`to_mcp`, `to_openapi`) be separate modules, or would the existing adapters gain serving capabilities?
## Scope
Design-only task. Produce:
1. A decision document (ADR or architecture section) answering the questions above
2. An API sketch showing the serving adapter interfaces
3. Identification of implementation work needed
## Constraints
- Must work with the existing `ResponseEnvelope` model — don't change the envelope to accommodate serving
- Must handle all three operation types (QUERY, MUTATION, SUBSCRIPTION)
- Runtime-agnostic (same as rest of package)
- Coherent with ADR-005 and ADR-006

View File

@@ -0,0 +1,158 @@
# Task: Implement ADR-007 Subscription Transport
**Priority**: High (blocks hub↔spoke streaming)
**Dependencies**: ADR-005 (done), ADR-006 (done). Heartbeat idle-timeout semantics settled in ADR-007 amended 2026-05-13.
**Architecture docs**: [ADR-007](../docs/architecture/decisions/007-subscription-transport.md), [call-protocol.md](../docs/architecture/call-protocol.md), [adapters.md](../docs/architecture/adapters.md)
## Scope
Three changes, all in source. No new modules needed.
### 1. `PendingRequestMap.subscribe()` — Remote subscription transport (src/call.ts)
Add a `subscribe()` method that returns `AsyncIterable<ResponseEnvelope>` using `@alkdev/pubsub`'s Repeater.
**What to build**:
1. Add `PendingEntry` discriminated union to track whether a `requestId` is a single-resolution `call` or a multi-yield `subscribe`:
```ts
type PendingEntry =
| { type: "call"; promise: PendingRequest }
| { type: "subscribe"; repeater: Repeater<ResponseEnvelope> }
```
Replace `Map<string, PendingRequest>` with `Map<string, PendingEntry>`.
2. Add `subscribe()` method on `PendingRequestMap`:
```ts
subscribe(
operationId: string,
input: unknown,
options?: { parentRequestId?: string; deadline?: number; identity?: Identity },
): AsyncIterable<ResponseEnvelope>
```
- Generate `requestId` via `crypto.randomUUID()`
- Create a `Repeater<ResponseEnvelope>` from `@alkdev/pubsub`
- Store `{ type: "subscribe", repeater }` in the map
- Publish `call.requested` (same as `call()` does)
- Return the Repeater as `AsyncIterable`
3. Update `setupSubscriptions()` event routing to handle both entry types:
- `call.responded:{requestId}`: If `type: "call"` → resolve promise + delete. If `type: "subscribe"` → push to repeater (keep entry).
- `call.error:{requestId}`: If `type: "call"` → reject promise + delete. If `type: "subscribe"` → push error to repeater then close repeater + delete.
- `call.aborted:{requestId}`: If `type: "call"` → reject promise + delete. If `type: "subscribe"` → close repeater + delete.
- Orphaned events (requestId not in map): silently ignore.
4. Implement heartbeat-based idle timeout for `subscribe()`:
- When `deadline` is set on `subscribe()`, store it in the `PendingEntry.subscribe`.
- In the `call.responded` handler for subscription entries, reset the deadline timer on each received envelope.
- If the deadline timer fires (no envelope received within `deadline` ms), treat it as `TIMEOUT` — close the repeater, publish `call.aborted`, delete the entry.
- Heartbeats from handlers are indistinguishable from data at the protocol level — a `ResponseEnvelope` with `_meta: { heartbeat: true }` is just another envelope that resets the deadline. Consumers filter heartbeats by inspecting `envelope._meta`.
5. Update `abort()` method: handle `type: "subscribe"` by closing the repeater.
6. Update `getPendingCount()`: count both call and subscribe entries.
### 2. CallHandler type dispatch (src/call.ts)
Update `buildCallHandler()` to check the operation's `type` and route accordingly:
- **QUERY / MUTATION**: Call `registry.execute()`, publish single `call.responded` or `call.error` (unchanged from current behavior)
- **SUBSCRIPTION**: Call `subscribe()`, iterate the generator, publish `call.responded` per yield, handle `call.aborted` by calling `generator.return()`
```ts
return async (event: CallRequestedEvent): Promise<void> => {
const { requestId, operationId, input, identity } = event;
const context: OperationContext = {
requestId,
parentRequestId: event.parentRequestId,
identity,
};
try {
const spec = registry.getSpec(operationId);
if (!spec) {
throw new CallError(InfrastructureErrorCode.OPERATION_NOT_FOUND, `Operation not found: ${operationId}`);
}
if (spec.type === OperationType.SUBSCRIPTION) {
// Subscription path: iterate generator, publish per yield
const generator = subscribe(registry, operationId, input, context);
for await (const envelope of generator) {
callMap.respond(requestId, envelope);
}
} else {
// QUERY / MUTATION: single response
const envelope = await registry.execute(operationId, input, context);
callMap.respond(requestId, envelope);
}
} catch (error) {
const spec = registry.getSpec(operationId);
const callError = mapError(error, spec?.errorSchemas);
callMap.emitError(requestId, callError.code, callError.message, callError.details);
}
};
```
For subscription abort handling: `CallHandler` should subscribe to `call.aborted:{requestId}` (or use the existing `callMap` mechanism) and call `generator.return()` when an abort arrives. This can be deferred to a follow-on — the initial implementation can let the generator naturally complete or error out.
### 3. FromOpenAPI SSE AsyncGenerator handlers (src/from_openapi.ts)
Replace the single-return `OperationHandler` for `SUBSCRIPTION`-type operations with an `AsyncGenerator`-based `SubscriptionHandler`.
**What to build**:
1. Add SSE parser function `parseSSEFrames(buffer: string): { events: SSEEvent[], remaining: string }`:
- Handle BOM (U+FEFF) at stream start
- Split on `\n`, `\r\n`, `\r`
- Parse `data:`, `event:`, `id:` fields per WHATWG spec
- `:`-prefixed lines (comments) are ignored
- Empty line dispatches buffered event
- Partial lines across `read()` calls: retain `remaining` buffer, prepend to next chunk
2. Add `SSEEvent` interface:
```ts
interface SSEEvent {
data: string
eventType: string
lastEventId: string
}
```
3. In `createHTTPOperation()`, when `opType === OperationType.SUBSCRIPTION`, generate a `SubscriptionHandler` instead of `OperationHandler`. The handler should:
- Call `fetch()` with URL/params/auth (same as current)
- On HTTP error, throw `CallError("EXECUTION_ERROR", ...)`
- Read response body as `ReadableStream` via `response.body.getReader()`
- Decode chunks with `TextDecoder`
- Parse SSE frames from the text stream via `parseSSEFrames()`
- Yield each parsed event wrapped in `httpEnvelope(data, { statusCode, headers, contentType: "text/event-stream" })`
- In `finally` block: release the reader lock
- Handle heartbeat yield if needed for long-lived SSE streams (TBD — initial pass can skip)
4. Update the return type of `createHTTPOperation()` and callers to handle the `OperationHandler | SubscriptionHandler` union:
- `FromOpenAPI()` return type: `Array<OperationSpec & { handler: OperationHandler | SubscriptionHandler }>`
- `FromOpenAPIFile()` and `FromOpenAPIUrl()` return types similarly
5. SSE `event` type and `id` fields: The parser parses them but they are NOT carried in the current `httpEnvelope()` — the `data` field value is the primary payload. If consumers need per-event metadata, a future `SSEResponseMeta` type can be added. This is out of scope for this task. Heartbeat detection (for filterable no-op envelopes) is also deferred.
## Tests
- `PendingRequestMap.subscribe()`: test with in-process EventTarget, verify Repeater yields each envelope, verify deadline kills after idle interval, verify abort cleans up, verify heartbeat resets deadline
- `CallHandler` dispatch: test QUERY/MUTATION path (execute), test SUBSCRIPTION path (subscribe + multiple responds), test error propagation
- SSE parser: test BOM, all line endings, multiple `data:` lines, `event:`/`id:` fields, comments, empty data, partial-line buffering, malformed lines
- `FromOpenAPI` SUBSCRIPTION handler: mock fetch returning a readable SSE stream, verify each event is yielded as httpEnvelope, verify error handling
## Acceptance Criteria
1. `PendingRequestMap.subscribe()` exists and returns `AsyncIterable<ResponseEnvelope>`
2. Multiple `call.responded` events with the same `requestId` route to the Repeater, not a single-resolution promise
3. Deadline timer fires as idle timeout (resets on each received envelope) for subscriptions
4. `CallHandler` dispatches QUERY/MUTATION to `execute()`, SUBSCRIPTION to `subscribe()`
5. SSE operations from `FromOpenAPI` are consumable as subscriptions
6. All existing tests still pass
7. TypeScript compiles clean (`npm run lint`)
## References
- `@alkdev/pubsub` Repeater API: see `src/repeater.ts` in `@alkdev/pubsub` source
- heartbeat idle timeout design: [ADR-007](../docs/architecture/decisions/007-subscription-transport.md) § Open Questions #2
- SSE spec: https://html.spec.whatwg.org/multipage/server-sent-events.html