From 51f233582d81601286637df334bd2ea347e4bff6 Mon Sep 17 00:00:00 2001 From: "glm-5.1" Date: Sun, 10 May 2026 08:37:40 +0000 Subject: [PATCH] Align call-protocol.md and api-surface.md with envelope model MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove supersession note from response-envelopes.md — both dependent docs now reflect the ResponseEnvelope system. Key changes: - call-protocol.md: CallHandler wraps and publishes (not handlers), call.responded.output uses ResponseEnvelopeSchema, respond() enforces envelope guard, call() resolves ResponseEnvelope, subscribe() yields ResponseEnvelope, references shared result pipeline - api-surface.md: execute() returns Promise>, OperationEnv functions return Promise, CallHandler calls handler directly and applies shared pipeline, respond() requires ResponseEnvelope, added Response Envelope Types and Utilities sections - response-envelopes.md: removed supersession note, added Shared Result Pipeline section (detect→wrap→normalize→validate), unified execute() and CallHandler integration points to reference shared pipeline, updated migration checklist to mark doc changes complete --- docs/architecture/api-surface.md | 85 ++++++++++++---- docs/architecture/call-protocol.md | 70 ++++++++----- docs/architecture/response-envelopes.md | 127 ++++++++++++++---------- 3 files changed, 186 insertions(+), 96 deletions(-) diff --git a/docs/architecture/api-surface.md b/docs/architecture/api-surface.md index fa31f88..9bd0760 100644 --- a/docs/architecture/api-surface.md +++ b/docs/architecture/api-surface.md @@ -1,11 +1,11 @@ --- status: draft -last_updated: 2026-05-09 +last_updated: 2026-05-10 --- # API Surface -All public types, registry, call protocol, subscribe, env, validation, and adapters. See [call-protocol.md](call-protocol.md) for detailed call protocol semantics and [adapters.md](adapters.md) for adapter internals. +All public types, registry, call protocol, subscribe, env, validation, adapters, and response envelopes. See [call-protocol.md](call-protocol.md) for detailed call protocol semantics, [response-envelopes.md](response-envelopes.md) for the envelope type system and integration points, and [adapters.md](adapters.md) for adapter internals. ## Core Types @@ -72,6 +72,41 @@ const ErrorDefinitionSchema = Type.Object({ Declared on `IOperationDefinition.errorSchemas`. Contract between operation and callers about what errors it may produce. +### Response Envelope Types + +All operation results are wrapped in `ResponseEnvelope` at the call protocol boundary. See [response-envelopes.md](response-envelopes.md) for the full type system, factory functions, and integration points. + +```ts +interface ResponseEnvelope { + data: T + meta: ResponseMeta +} + +type ResponseMeta = LocalResponseMeta | HTTPResponseMeta | MCPResponseMeta +type ResponseSource = "local" | "http" | "mcp" + +interface LocalResponseMeta { + source: "local" + operationId: string + timestamp: number +} + +interface HTTPResponseMeta { + source: "http" + statusCode: number + headers: Record + contentType: string +} + +interface MCPResponseMeta { + source: "mcp" + isError: boolean + content: MCPContentBlock[] + structuredContent?: Record + _meta?: Record +} +``` + ### `OperationContext` ```ts @@ -127,15 +162,17 @@ type SubscriptionHandler = ( ) => AsyncGenerator ``` -`OperationHandler` returns a single value. `SubscriptionHandler` yields values over time. +`OperationHandler` returns a single value. `SubscriptionHandler` yields values over time. Both return/yield **raw values** — wrapping in `ResponseEnvelope` happens in infrastructure (`execute()`, `CallHandler`, `subscribe()`). Handlers that need to provide transport metadata can return a pre-built `ResponseEnvelope` (detected by `isResponseEnvelope()`), but this is only needed for adapter handlers (MCP, OpenAPI). ### `OperationEnv` ```ts -type OperationEnv = Record Promise>> +type OperationEnv = Record Promise>> ``` -Namespace-keyed operation map. Accessed as `env.namespace.operationName(input)`. Created by `buildEnv`. +Namespace-keyed operation map. Accessed as `env.namespace.operationName(input)`. Created by `buildEnv`. Each inner function returns `Promise` — callers access typed data via `envelope.data` and metadata via `envelope.meta`, or use `unwrap(envelope)` for the common case where only `data` is needed. + +**Type note**: `OperationEnv` inner functions return `Promise` (untyped), which means callers lose per-operation type inference through `OperationEnv`. The generic `TOutput` of the underlying operation is not propagated through the namespace-keyed map — this is an inherent limitation of the string-keyed access pattern. Consumers should use `envelope.data` with their own type narrowing, or use `registry.execute()` directly when type inference is needed. ## Registry @@ -155,13 +192,13 @@ The registry stores specs and handlers in separate internal maps. Specs are seri | `getByName(namespace, name)` | `(namespace: string, name: string) => (OperationSpec & { handler?: ... }) \| undefined` | Get by parts. | | `list()` | `() => Array` | All registered entries (spec + handler if present). | | `getAllSpecs()` | `() => OperationSpec[]` | All serializable specs. | -| `execute(operationId, input, context)` | `(id: string, input: TInput, ctx: OperationContext) => Promise` | Validate input, run handler, warn on output mismatch. Throws if spec or handler not found. | +| `execute(operationId, input, context)` | `(id: string, input: TInput, ctx: OperationContext) => Promise>` | Validate input, run handler, wrap result in `ResponseEnvelope`, warn on output mismatch. Throws if spec or handler not found. | Registration key format: `{namespace}.{name}`. Overwrite on duplicate. Specs and handlers can be registered independently: `registerSpec()` then `registerHandler()` for the same id, or `register()` with `{ ...spec, handler }` in one call. `execute()` requires both — throws `"Operation not found"` if spec missing, `"No handler registered"` if handler missing. -`execute` validates input with `validateOrThrow` before calling the handler. Output validation uses `collectErrors` and logs warnings — it does not throw. +`execute` validates input with `validateOrThrow` before calling the handler. The handler return value is wrapped in a `ResponseEnvelope` via `isResponseEnvelope()` detection — if the result is already an envelope, it passes through; otherwise `localEnvelope(result, operationId)` wraps it. Output validation uses `collectErrors` on `envelope.data` against `spec.outputSchema` and logs warnings — it does not throw. ## Call Protocol @@ -172,8 +209,8 @@ See [call-protocol.md](call-protocol.md) for full semantics. | Method | Signature | Description | |--------|-----------|-------------| | `constructor(eventTarget?)` | `(eventTarget?: EventTarget)` | Creates internal pubsub, wires subscription handlers for responded/error/aborted. | -| `call(operationId, input, options?)` | `Promise` | Publish `call.requested`, return Promise that resolves on `call.responded`. | -| `respond(requestId, output)` | `void` | Publish `call.responded`. | +| `call(operationId, input, options?)` | `Promise` | Publish `call.requested`, return Promise that resolves with `ResponseEnvelope` on `call.responded`. | +| `respond(requestId, output)` | `void` | Publish `call.responded`. `output` must be `ResponseEnvelope` — `isResponseEnvelope()` guard throws on raw values. | | `emitError(requestId, code, message, details?)` | `void` | Publish `call.error`. | | `abort(requestId)` | `void` | Publish `call.aborted`, reject pending Promise. | | `getPendingCount()` | `number` | Number of in-flight requests. | @@ -184,7 +221,7 @@ See [call-protocol.md](call-protocol.md) for full semantics. type CallHandler = (event: CallRequestedEvent) => Promise ``` -Created by `buildCallHandler({ registry, eventTarget? })`. Subscribes to `call.requested`, checks access control, validates input, executes via registry. On success: no-op (handler is expected to publish `call.responded` through the PendingRequestMap). On failure: throws `CallError`. +Created by `buildCallHandler({ registry, eventTarget? })`. Subscribes to `call.requested`, checks access control, validates input, calls the handler directly (not via `registry.execute()`), applies the shared result pipeline (detect → wrap → normalize → validate), and publishes `call.responded`. On failure: publishes `call.error` with mapped `CallError`. Adapters that return pre-built envelopes (MCP, OpenAPI) pass through via `isResponseEnvelope()` detection. See [response-envelopes.md](response-envelopes.md#shared-result-pipeline) for the shared pipeline definition. ### `CallEventMap` @@ -204,7 +241,7 @@ Typed event map compatible with `@alkdev/pubsub`. See [call-protocol.md](call-pr | Type | Fields | Description | |------|--------|-------------| | `CallRequestedEvent` | `requestId, operationId, input, parentRequestId?, deadline?, identity?` | Initiates a call | -| `CallRespondedEvent` | `requestId, output` | Successful response | +| `CallRespondedEvent` | `requestId, output: ResponseEnvelope` | Successful response (envelope always present) | | `CallAbortedEvent` | `requestId` | Call cancelled | | `CallErrorEvent` | `requestId, code, message, details?` | Error response | @@ -213,15 +250,15 @@ Typed event map compatible with `@alkdev/pubsub`. See [call-protocol.md](call-pr ### `subscribe` ```ts -function subscribe( +async function* subscribe( registry: OperationRegistry, operationId: string, input: unknown, context: OperationContext, -): AsyncGenerator +): AsyncGenerator ``` -Direct subscription execution. Gets the operation, casts its handler to `AsyncGenerator`, yields each value. Properly cleans up the generator on iteration stop (calls `generator.return()` in `finally`). +Direct subscription execution. Gets the operation, casts its handler to `AsyncGenerator`, yields each value wrapped in `ResponseEnvelope`. If a yielded value is already an envelope (`isResponseEnvelope()`), it passes through. Otherwise, `localEnvelope(value, operationId)` wraps it. Properly cleans up the generator on iteration stop (calls `generator.return()` in `finally`). This is the synchronous alternative to the call protocol's `call.requested` → `call.responded` flow for subscriptions. Use `subscribe()` for in-process subscription consumption; use `PendingRequestMap` for cross-transport subscription. @@ -240,10 +277,10 @@ interface EnvOptions { } ``` -Creates a namespace-keyed `OperationEnv` for nested operation calls. Two modes: +Creates a namespace-keyed `OperationEnv` for nested operation calls. Each env function returns `Promise` — callers access typed data via `envelope.data` or use `unwrap(envelope)`. Two modes: -- **Direct mode**: `buildEnv({ registry, context })` — env functions call `registry.execute()` -- **Call protocol mode**: `buildEnv({ registry, context, callMap })` — env functions call `callMap.call()`, publishing `call.requested` events with `parentRequestId` for call graph tracking +- **Direct mode**: `buildEnv({ registry, context })` — env functions call `registry.execute()`, which wraps in `localEnvelope` +- **Call protocol mode**: `buildEnv({ registry, context, callMap })` — env functions call `callMap.call()`, which resolves to `ResponseEnvelope` directly, publishing `call.requested` events with `parentRequestId` for call graph tracking `SUBSCRIPTION` operations are filtered out — env only provides QUERY and MUTATION operations for nested calls. @@ -306,6 +343,20 @@ Converts JSON Schema to TypeBox `TSchema`. Handles: `allOf`, `anyOf`, `oneOf`, ` Used internally by `FromOpenAPI` to convert OpenAPI JSON Schema definitions to TypeBox. Also used by `from_mcp` to convert MCP tool `inputSchema` (which is JSON Schema). +## Response Envelope Utilities + +See [response-envelopes.md](response-envelopes.md) for detailed semantics and integration points. + +| Export | Signature | Description | +|--------|-----------|-------------| +| `isResponseEnvelope(value)` | `(unknown) => value is ResponseEnvelope` | Type guard. Checks `meta.source` discriminant against `"local" \| "http" \| "mcp"`. | +| `localEnvelope(data, operationId)` | `(data: T, operationId: string) => ResponseEnvelope` | Wrap local handler result. | +| `httpEnvelope(data, meta)` | `(data: T, meta: Omit) => ResponseEnvelope` | Wrap HTTP response data. | +| `mcpEnvelope(data, meta)` | `(data: T, meta: Omit) => ResponseEnvelope` | Wrap MCP tool result. | +| `unwrap(envelope)` | `(envelope: ResponseEnvelope) => T` | Convenience: returns `envelope.data`. | +| `ResponseEnvelopeSchema` | `TSchema` | TypeBox schema for `ResponseEnvelope`. | +| `ResponseMetaSchema` | `TSchema` | TypeBox schema for the `ResponseMeta` discriminated union. | + ## Adapters See [adapters.md](adapters.md) for detailed adapter documentation. diff --git a/docs/architecture/call-protocol.md b/docs/architecture/call-protocol.md index 059a5ce..7640fc7 100644 --- a/docs/architecture/call-protocol.md +++ b/docs/architecture/call-protocol.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-05-09 +last_updated: 2026-05-10 --- # Call Protocol @@ -13,10 +13,10 @@ The call protocol is the unified transport layer for all operation invocations. At the protocol level, `call` and `subscribe` are the same thing with different consumption patterns: -- **`call`**: Publish `call.requested`, subscribe to `call.responded:{requestId}`, resolve on first response → `Promise` -- **`subscribe`**: Publish `call.requested`, subscribe to `call.responded:{requestId}`, yield each response → `AsyncIterable` +- **`call`**: Publish `call.requested`, subscribe to `call.responded:{requestId}`, resolve on first response → `Promise` +- **`subscribe`**: Publish `call.requested`, subscribe to `call.responded:{requestId}`, yield each response → `AsyncIterable` -Both use the same event types, the same `requestId` correlation, and the same `PendingRequestMap`. `call` is semantically `subscribe().next()`. +Both use the same event types, the same `requestId` correlation, and the same `PendingRequestMap`. `call` is semantically `subscribe().next()`. All responses are wrapped in `ResponseEnvelope` — see [response-envelopes.md](response-envelopes.md) for the full envelope type system. ## Event Types @@ -40,7 +40,7 @@ const CallEventMap = { }), "call.responded": Type.Object({ requestId: Type.String(), - output: Type.Unknown(), + output: ResponseEnvelopeSchema, }), "call.aborted": Type.Object({ requestId: Type.String(), @@ -54,6 +54,8 @@ const CallEventMap = { } ``` +`call.responded.output` uses `ResponseEnvelopeSchema` (defined in [response-envelopes.md](response-envelopes.md)). This means every response through the call protocol carries `data` and `meta` with source-discriminated metadata. Handlers do not construct this envelope manually — `CallHandler` wraps handler return values automatically. + ### Request Correlation Every call has a unique `requestId` (UUID). Nested calls include `parentRequestId` to track the call chain. Responses and errors match to requests by `requestId`. @@ -66,9 +68,11 @@ Caller Handler │─── call.requested ───────────────>│ │ {requestId, operationId, │ │ input, identity, deadline} │ - │ │ + │ │ handler returns value + │ │ CallHandler wraps in ResponseEnvelope │<── call.responded ────────────────│ - │ {requestId, output} │ + │ {requestId, │ + │ output: ResponseEnvelope} │ ``` On error: @@ -112,7 +116,7 @@ async call( operationId: string, input: unknown, options?: { parentRequestId?: string; deadline?: number; identity?: Identity }, -): Promise +): Promise ``` 1. Generate `requestId` via `crypto.randomUUID()` @@ -120,19 +124,23 @@ async call( 3. If `deadline` is set, start a timeout timer that rejects with `TIMEOUT` 4. Store `PendingRequest` in the internal map 5. Publish `call.requested` event with all fields -6. Return the Promise (resolves on `call.responded`, rejects on `call.error` or `call.aborted`) +6. Return the Promise (resolves with `ResponseEnvelope` on `call.responded`, rejects on `call.error` or `call.aborted`) + +The resolved value is a `ResponseEnvelope` — consumers access typed data via `envelope.data` and source metadata via `envelope.meta`. Use `unwrap(envelope)` as a convenience for the common case where only `data` is needed. ### Internal Subscription Wiring On construction, three async loops subscribe to pubsub topics: -- **`call.responded`**: Look up `PendingRequest` by `requestId`, clear timer if set, resolve with `output` +- **`call.responded`**: Look up `PendingRequest` by `requestId`, clear timer if set, resolve with the `ResponseEnvelope` from `output` field. The envelope is already validated by `respond()`'s `isResponseEnvelope()` guard (or created by `CallHandler`'s wrapping logic), so no additional validation is needed at this point. - **`call.error`**: Look up `PendingRequest`, clear timer, reject with `CallError(code, message, details)` - **`call.aborted`**: Look up `PendingRequest`, clear timer, reject with `CallError(ABORTED, ...)` ### `respond(requestId, output)` -Publishes `call.responded`. Used by handlers to send results back through the protocol. +Publishes `call.responded`. The `output` parameter must be a `ResponseEnvelope` — `isResponseEnvelope()` is checked and a non-envelope value throws. This enforces the invariant that all call protocol responses carry source metadata. + +In practice, `respond()` is called by `CallHandler` after wrapping the handler's return value. Direct calls to `respond()` with raw values are rejected. ### `emitError(requestId, code, message, details?)` @@ -144,7 +152,7 @@ Looks up the `PendingRequest`, clears its timer, publishes `call.aborted`, rejec ## CallHandler -`buildCallHandler` creates a function that bridges pubsub events to `OperationRegistry.execute()`. +`buildCallHandler` creates a function that bridges pubsub events to `OperationRegistry.execute()`. It takes full ownership of publishing `call.responded` — handlers return values; they do NOT publish events. ```ts function buildCallHandler(config: CallHandlerConfig): CallHandler @@ -166,21 +174,26 @@ type CallHandler = (event: CallRequestedEvent) => Promise 5. Check access control (see below) 6. Validate input with `validateOrThrow` 7. Execute operation handler -8. On success: the handler is expected to have published `call.responded` through whatever mechanism -9. On failure: `mapError` converts the thrown value to `CallError` +8. On success: apply the shared result pipeline (see [Response Envelopes → Shared Result Pipeline](response-envelopes.md#shared-result-pipeline)): + - Detect: `isResponseEnvelope(result)` → pass through, otherwise `localEnvelope(result, operationId)` + - Normalize: `Value.Cast(spec.outputSchema, envelope.data)` when `outputSchema` is not `Type.Unknown()` + - Validate: `collectErrors(spec.outputSchema, envelope.data)` — warning-only + - Publish `call.responded` via `callMap.respond(requestId, envelope)` +9. On failure: `mapError` converts the thrown value to `CallError`, publish `call.error` -The `CallHandler` is designed to be wired into a pubsub subscription: +**Key change**: In the pre-envelope model, handlers were responsible for publishing `call.responded` themselves (the handler return value was discarded). In the envelope model, `CallHandler` owns wrapping and publishing. Handler return values are captured and wrapped. This ensures every response goes through the envelope pipeline — no raw values can bypass it. -```ts -const callHandler = buildCallHandler({ registry, eventTarget }) -pubsub.subscribe("call.requested", callHandler) -``` +### MCP and OpenAPI Handlers + +Adapter handlers (from `from_mcp` and `from_openapi`) return pre-built `ResponseEnvelope` instances via `mcpEnvelope()` and `httpEnvelope()` factory functions. When `CallHandler` detects `isResponseEnvelope()` on the result, it passes through without re-wrapping. This means adapter metadata (HTTP status codes, MCP `isError` flags) is preserved. + +For MCP results with `meta.isError: true`, the handler still returns an envelope — the error is represented as data, not thrown. Only thrown exceptions trigger `call.error`. ## Access Control ### Enforcement Point -`CallHandler` enforces `AccessControl` before dispatching to `registry.execute()`. Direct `registry.execute()` calls bypass access control — this is by design for trusted internal calls. +`CallHandler` enforces `AccessControl` before calling the handler directly. Direct `registry.execute()` calls bypass access control — this is by design for trusted internal calls. ### Flow @@ -251,8 +264,8 @@ Operations declare their possible errors via `errorSchemas` on `IOperationDefini Routing is an env construction concern, not a separate protocol layer. `buildEnv` creates the `OperationEnv`: -- **Direct mode**: `buildEnv({ registry, context })` — env functions call `registry.execute()` directly -- **Call protocol mode**: `buildEnv({ registry, context, callMap })` — env functions call `callMap.call()`, publishing `call.requested` events with `parentRequestId` propagation +- **Direct mode**: `buildEnv({ registry, context })` — env functions call `registry.execute()` directly, returning `Promise` +- **Call protocol mode**: `buildEnv({ registry, context, callMap })` — env functions call `callMap.call()`, which resolves to `Promise`, publishing `call.requested` events with `parentRequestId` propagation `parentRequestId` enables call graph reconstruction and abort cascading — every nested call includes it. @@ -278,10 +291,10 @@ async function* subscribe( operationId: string, input: unknown, context: OperationContext, -): AsyncGenerator +): AsyncGenerator ``` -Gets the operation from the registry, casts its handler to `AsyncGenerator`, and yields values. Properly cleans up with `generator.return()` in a `finally` block. +Gets the operation from the registry, casts its handler to `AsyncGenerator`, and yields each value wrapped in `ResponseEnvelope`. If a yielded value `isResponseEnvelope()`, it passes through (e.g., for adapter handlers). Otherwise, `localEnvelope(value, operationId)` wraps it with a fresh `timestamp` per yield. Properly cleans up with `generator.return()` in a `finally` block. Use `subscribe()` for in-process consumption. Use `PendingRequestMap.call()` for cross-transport invocation that resolves after one event. For cross-transport streaming, use `PendingRequestMap.subscribe()` to yield multiple events. @@ -292,4 +305,11 @@ The `subscribe()` function looks up both spec and handler separately from the re 1. `registry.getSpec(operationId)` — throws if spec not found 2. `registry.getHandler(operationId)` — throws if handler not found -This allows spec-only registration for scenarios where handlers are provided separately (e.g., ujsx host interpretation, dynamic handler injection). \ No newline at end of file +This allows spec-only registration for scenarios where handlers are provided separately (e.g., ujsx host interpretation, dynamic handler injection). + +## References + +- [response-envelopes.md](response-envelopes.md) — `ResponseEnvelope` types, factory functions, detection, and integration points +- [ADR-005](decisions/005-response-envelopes.md) — Design rationale for response envelopes +- [api-surface.md](api-surface.md) — Public API surface (types and signatures) +- [adapters.md](adapters.md) — MCP and OpenAPI adapter internals \ No newline at end of file diff --git a/docs/architecture/response-envelopes.md b/docs/architecture/response-envelopes.md index f784577..c08cfd2 100644 --- a/docs/architecture/response-envelopes.md +++ b/docs/architecture/response-envelopes.md @@ -5,8 +5,6 @@ last_updated: 2026-05-10 # Response Envelopes -> **Note**: This spec supersedes the current behavior described in [call-protocol.md](call-protocol.md) and [api-surface.md](api-surface.md). Those documents describe the pre-envelope model where `execute()` returns `TOutput` and handlers publish `call.responded` directly. The migration checklist below tracks the required updates. Until those documents are updated, this spec is the authoritative source for envelope behavior. - Types, factory functions, integration points, and constraints for the `ResponseEnvelope` system. See [ADR-005](decisions/005-response-envelopes.md) for the design rationale. ## Problem @@ -190,6 +188,26 @@ Individual meta schemas (`LocalResponseMetaSchema`, `HTTPResponseMetaSchema`, `M ## Integration Points +### Shared Result Pipeline + +Both `OperationRegistry.execute()` and `CallHandler` follow the same result processing pipeline after obtaining a handler result. This ensures consistent behavior whether the operation is invoked directly or through the call protocol: + +1. **Detect envelope**: If result `isResponseEnvelope()` → pass through as-is (adapter handlers return pre-built envelopes) +2. **Wrap**: Otherwise → wrap in `localEnvelope(result, operationId)` +3. **Normalize**: If `spec.outputSchema` is not `Type.Unknown()`, apply `Value.Cast(spec.outputSchema, envelope.data)` — strips excess properties, fills defaults, upcasts values +4. **Validate**: Check `envelope.data` against `spec.outputSchema` with `collectErrors()` — **warning-only**, logged but not thrown + +The order matters: normalization happens before validation, so warnings reflect the normalized data shape. `Value.Cast()` may silently fix data that would otherwise fail validation (e.g., filling defaults for optional fields), and the validation step catches any remaining mismatches. + +### Result Pipeline Differences + +| Aspect | `execute()` | `CallHandler` | +|--------|-------------|---------------| +| Access control | Not checked (trusted internal calls) | Checks `AccessControl` before dispatch | +| Error handling | Throws `CallError` directly | Publishes `call.error` via pubsub | +| Publishing | Returns envelope directly | Publishes `call.responded` via `callMap.respond()` | +| Context | Direct call, no `requestId` | Has `requestId`, `parentRequestId`, `identity`, `deadline` | + ### `OperationRegistry.execute()` Returns `Promise>` instead of `Promise`. @@ -198,10 +216,8 @@ Flow: 1. Look up spec and handler (existing) 2. Validate input with `validateOrThrow` (existing) 3. Await handler result -4. If result `isResponseEnvelope()` → pass through as-is -5. Otherwise → wrap in `localEnvelope(result, operationId)` -6. Validate `envelope.data` against `spec.outputSchema` — warning-only, logged but not thrown -7. Return envelope +4. Apply shared result pipeline (detect → wrap → normalize → validate) +5. Return envelope **Note**: `isResponseEnvelope()` does not validate that the envelope's `source` matches the operation's origin. An MCP handler that explicitly returns a `localEnvelope(...)` passes through as-is. Handlers that explicitly construct envelopes take responsibility for their metadata. @@ -219,15 +235,16 @@ Flow: Takes full ownership of publishing `call.responded`. Handlers return values; they do NOT publish events. Flow: -1. Look up spec and handler (existing) -2. Check access control (existing) -3. Validate input (existing) -4. Call handler and await result -5. If result `isResponseEnvelope()` → use directly -6. Otherwise → wrap in `localEnvelope(result, operationId)` -7. Validate `envelope.data` against `spec.outputSchema` — warning-only -8. Publish `call.responded` via `callMap.respond(requestId, envelope)` -9. On handler exception → publish `call.error` (existing). Note: an envelope with `meta.isError: true` does **not** trigger `call.error`. Only thrown exceptions do. +1. Look up spec by `operationId` from the registry via `getSpec()` +2. If not found, throw `CallError(OPERATION_NOT_FOUND, ...)` +3. Look up handler by `operationId` via `getHandler()` +4. If not found, throw `CallError(OPERATION_NOT_FOUND, "No handler registered for operation: ...")` +5. Check access control (existing) +6. Validate input with `validateOrThrow` (existing) +7. Call handler and await result +8. Apply shared result pipeline (detect → wrap → normalize → validate) +9. Publish `call.responded` via `callMap.respond(requestId, envelope)` +10. On handler exception → publish `call.error` (existing). Note: an envelope with `meta.isError: true` does **not** trigger `call.error`. Only thrown exceptions do. **Current source state** (`src/call.ts` lines 182-233): `buildCallHandler` calls `handler(input, context)` (line 226) but does not use the return value. The handler is expected to publish `call.responded` itself. Changes needed: @@ -236,6 +253,7 @@ Flow: | Handler model | Handler publishes `call.responded` itself; return value ignored | Handler returns value; `CallHandler` wraps and publishes | | Return value | `await handler(input, context)` called but result discarded | Result captured, wrapped in envelope if needed, then published | | Envelope detection | Not applicable | `isResponseEnvelope(result)` check before wrapping | +| Result pipeline | None | Detect → wrap → normalize → validate → publish | | `call.responded.output` | `Type.Unknown()` | `ResponseEnvelopeSchema` | | `PendingRequestMap.respond()` | Accepts any `unknown` value | Must enforce `isResponseEnvelope()` guard | @@ -366,27 +384,13 @@ handler: async (input, context) => { ### Value.Cast() for Data Normalization -When an adapter has a meaningful `outputSchema` (not `Type.Unknown()`), `Value.Cast()` from `@alkdev/typebox/value` can normalize `envelope.data` against the schema: +The shared result pipeline (defined above) includes `Value.Cast()` normalization as step 3. This section provides additional context on how `Value.Cast()` works and why it matters for each source: -```ts -import { Value } from "@alkdev/typebox/value" +- **Local operations**: `Value.Cast()` provides normalization against the declared `outputSchema`, stripping excess properties and filling defaults — a stronger guarantee than `Value.Check()` validation alone (which only warns). +- **MCP operations with `outputSchema`**: `Value.Cast()` normalizes `structuredContent` against the TypeBox-converted schema, stripping any extra properties the MCP server may have added beyond the declared schema. This makes MCP data composable with local operations. +- **OpenAPI operations**: `Value.Cast()` normalizes the parsed HTTP response body against the operation's `outputSchema`, ensuring consistent data shapes. -// In execute(), after wrapping the result: -if (!isResponseEnvelope(result)) { - envelope = localEnvelope(result, operationId) -} else { - envelope = result -} - -// Normalize data against outputSchema (when schema is meaningful) -if (spec.outputSchema[Kind] !== "Unknown") { - envelope.data = Value.Cast(spec.outputSchema, envelope.data) -} -``` - -This strips excess properties, fills defaults for missing ones, and upcasts values to match the declared type. For local operations, this provides the same guarantee as `Value.Check()` validation but with normalization instead of just warning. For MCP operations with `outputSchema`, it strips envelope-like properties that MCP servers might add beyond the declared schema. For OpenAPI operations, it normalizes the response body against the spec. - -Operations with `Type.Unknown()` `outputSchema` (typically MCP tools without `outputSchema`) are excluded — `Value.Cast()` against `Type.Unknown()` is a no-op. +Operations with `Type.Unknown()` `outputSchema` (typically MCP tools without `outputSchema`) are excluded — `Value.Cast()` against `Type.Unknown()` is a no-op (accepts any value). ### `CallEventSchema` @@ -403,12 +407,12 @@ Operations with `Type.Unknown()` `outputSchema` (typically MCP tools without `ou `outputSchema` on `OperationSpec` validates `envelope.data`, not the full `ResponseEnvelope`. The envelope has its own `ResponseEnvelopeSchema` for call protocol validation. This keeps `outputSchema` focused on business data shape — no envelope schema bloat, and existing `outputSchema` definitions don't need changes. -There are two validation/normalization steps for `envelope.data`: +The shared result pipeline (see [Shared Result Pipeline](#shared-result-pipeline)) applies two steps to `envelope.data` after wrapping: -1. **Warning validation** (`collectErrors` + `formatValueErrors`): Checks `envelope.data` against `spec.outputSchema` and logs warnings on mismatch. This is the existing behavior in `execute()`. -2. **Data normalization** (`Value.Cast`): When `outputSchema` is meaningful (not `Type.Unknown()`), `Value.Cast(spec.outputSchema, envelope.data)` normalizes the data — strips excess properties, fills defaults for missing ones, upcasts values to match the declared type. This is particularly important for MCP `structuredContent` (which may contain extra properties from the MCP envelope) and OpenAPI response bodies (which may have extra fields not in the spec). +1. **Normalization** (`Value.Cast`): When `outputSchema` is meaningful (not `Type.Unknown()`), `Value.Cast(spec.outputSchema, envelope.data)` normalizes the data — strips excess properties, fills defaults for missing ones, upcasts values to match the declared type. Normalization happens before validation. +2. **Warning validation** (`collectErrors` + `formatValueErrors`): Checks `envelope.data` against `spec.outputSchema` and logs warnings on mismatch. Validation happens after normalization, so it checks the normalized data shape. -The `Value.Cast()` step is optional: if `outputSchema` is `Type.Unknown()`, normalization is skipped (any value is accepted). This preserves the current behavior for MCP tools without `outputSchema`. +The normalization step is optional: if `outputSchema` is `Type.Unknown()`, normalization is skipped (any value is accepted). This preserves the current behavior for MCP tools without `outputSchema`. ## Constraints @@ -437,23 +441,38 @@ The `Value.Cast()` step is optional: if `outputSchema` is `Type.Unknown()`, norm ## Migration Checklist -When this spec stabilizes, the following documents and code must be updated: +The following documentation changes have been completed: -| Document | Section | Change | -|----------|---------|--------| -| `call-protocol.md` | CallHandler | Handler no longer publishes `call.responded`; returns values. CallHandler wraps and publishes. | -| `call-protocol.md` | PendingRequestMap | `respond()` validates envelope via `isResponseEnvelope()`; resolves with `ResponseEnvelope` instead of `unknown` | -| `call-protocol.md` | CallEventMap | `call.responded.output` changes from `Type.Unknown()` to `ResponseEnvelopeSchema` | -| `api-surface.md` | `execute()` | Return type: `Promise` → `Promise>` | -| `api-surface.md` | `PendingRequestMap.call()` | Resolve type: `unknown` → `ResponseEnvelope` | -| `api-surface.md` | `subscribe()` | Yield type: `unknown` → `ResponseEnvelope` | -| `api-surface.md` | `OperationEnv` | Inner function return type: `Promise` → `Promise` | -| `api-surface.md` | `CallHandler` | New: wraps handler result, publishes `call.responded`. No longer "handler publishes" model. | -| `call-protocol.md` | `PendingRequestMap.respond()` | Now enforces `isResponseEnvelope()` check — throws on raw values. Behavioral breaking change. | -| `api-surface.md` | `PendingRequestMap.respond()` | Same — `respond()` now requires `ResponseEnvelope` argument. | -| `adapters.md` | `from_mcp` | Handler returns `mcpEnvelope()`. MCP `isError: true` no longer throws. | -| `adapters.md` | `from_mcp` | `outputSchema` extracted from MCP tool definitions when available (2025-06-18+ spec), converted via `FromSchema`. Falls back to `Type.Unknown()`. | -| `adapters.md` | `from_openapi` | Handler returns `httpEnvelope()`. Error on HTTP error status still throws `CallError`. | +| Document | Section | Change | Status | +|----------|---------|--------|--------| +| `call-protocol.md` | CallHandler | Handler no longer publishes `call.responded`; returns values. CallHandler wraps and publishes. | ✅ | +| `call-protocol.md` | PendingRequestMap | `respond()` validates envelope via `isResponseEnvelope()`; resolves with `ResponseEnvelope` instead of `unknown` | ✅ | +| `call-protocol.md` | CallEventMap | `call.responded.output` changes from `Type.Unknown()` to `ResponseEnvelopeSchema` | ✅ | +| `api-surface.md` | `execute()` | Return type: `Promise` → `Promise>` | ✅ | +| `api-surface.md` | `PendingRequestMap.call()` | Resolve type: `unknown` → `ResponseEnvelope` | ✅ | +| `api-surface.md` | `subscribe()` | Yield type: `unknown` → `ResponseEnvelope` | ✅ | +| `api-surface.md` | `OperationEnv` | Inner function return type: `Promise` → `Promise` | ✅ | +| `api-surface.md` | `CallHandler` | Wraps handler result, publishes `call.responded`. No longer "handler publishes" model. | ✅ | +| `call-protocol.md` | `PendingRequestMap.respond()` | Now enforces `isResponseEnvelope()` check — throws on raw values. | ✅ | +| `api-surface.md` | `PendingRequestMap.respond()` | `respond()` now requires `ResponseEnvelope` argument. | ✅ | +| `adapters.md` | `from_mcp` | Handler returns `mcpEnvelope()`. MCP `isError: true` no longer throws. | ✅ (previous) | +| `adapters.md` | `from_mcp` | `outputSchema` extracted when available, via `FromSchema`. Falls back to `Type.Unknown()`. | ✅ (previous) | +| `adapters.md` | `from_openapi` | Handler returns `httpEnvelope()`. Error on HTTP error status still throws `CallError`. | ✅ (previous) | + +The following **code** changes are still needed: + +| Code | Change | +|------|--------| +| `src/registry.ts` | `execute()` returns `Promise>` | +| `src/call.ts` | `CallHandler` captures return value, wraps in envelope, publishes `call.responded` | +| `src/call.ts` | `CallEventSchema` `output` field changes to `ResponseEnvelopeSchema` | +| `src/call.ts` | `PendingRequestMap.respond()` adds `isResponseEnvelope()` guard | +| `src/call.ts` | `PendingRequestMap.call()` resolves with `ResponseEnvelope` | +| `src/subscribe.ts` | `subscribe()` wraps yields in `ResponseEnvelope` | +| `src/env.ts` | `buildEnv()` functions return `Promise` | +| `src/response-envelope.ts` | New file: types, factory functions, detection, schemas | +| `src/from_mcp.ts` | Handler returns `mcpEnvelope()`, extracts `outputSchema`, uses `structuredContent` | +| `src/from_openapi.ts` | Handler returns `httpEnvelope()` | Additionally, any code subscribing to `"call.responded"` events via the pubsub system (not just `PendingRequestMap`, but any direct pubsub consumer) must expect `ResponseEnvelope` instead of `unknown` in the event payload.