alkhttp/docs/architecture/decisions/068-gateway-publish-endpoint.md

ADR-068: Gateway /publish Endpoint for Pub Operations

Status

Accepted

Context

alkcall added OperationType::Pub and HandlerKind::Sink (alkcall ADR-046): producer→consumer streaming, where the initiator streams chunks to the operation via call.published events and the handler consumes them as a PublishStream. This is the inverse of Sub (consumer→producer streaming of results) and completes the four operation types: Query, Mutation, Sub, Pub.

The HTTP gateway (ADR-042, ADR-047) exposes five endpoints: /search, /schema, /call, /batch, /subscribe. The gateway's dispatch covers Query/Mutation (/call, /batch) and Sub (/subscribe via the SSE projection, ADR-049). Pub operations have no HTTP expression: an HTTP client cannot feed a Pub operation's sink.

Without a surface, Pub operations are call-protocol-only (WS channel 0 or a QUIC/alk/channels session). That leaves HTTP clients — curl, axios, a server-side script — unable to produce into Pub ops, which breaks parity: every operation type reachable over the call protocol should be reachable over HTTP, or the HTTP surface is not a faithful projection.

Decision

The gateway gains a sixth endpoint: POST /publish. It invokes a Pub operation; the HTTP request body is the initiator's publish stream; the operation's final ResponseEnvelope is the HTTP response.

Request

• Path: POST /publish.
• Body: newline-delimited JSON (NDJSON) — each line is one published chunk, serialized as a JSON value; the stream of lines maps 1:1 to call.published events. Chunk boundaries are the line boundaries; a line's JSON value is the chunk payload.
• Auth: Authorization: Bearer <token> — same as every gateway endpoint (ADR-004).
• The target operation is named the same way as /call — the body's first line carries { "operation": "/{service}/{op}", "chunk": {...} for the first chunk, with subsequent lines carrying chunk values only. (OQ-02 resolved: first-line convention, no query parameter, no header; a ?operation= parameter was rejected because it duplicates the first-line field and complicates curl one-liners for no gain.)

Dispatch

1. Resolve identity (Bearer → resolve_from_token).
2. Look up the operation; enforce Visibility::External (Internal → 404, same as /call) and AccessControl::check (→ 403).
3. Verify op_type == Pub — a non-Pub op is INVALID_OPERATION_TYPE → 400.
4. Dispatch through invoke_sink() (alkcall ADR-046): stream each NDJSON line as one call.published chunk into the handler's PublishStream.
5. On end-of-body, deliver the handler's final ResponseEnvelope:
   • Ok(output) → 200 with the output as JSON.
   • Err(call_error) → mapped status per the standard error mapping (ADR-023, the gateway's HTTP_<status> fidelity rules).

Body handling (streamed, not buffered)

The NDJSON body is streamed, never fully buffered (GW-06): axum's Body is framed into lines as bytes arrive, and each line is parsed and pushed into the sink lazily — memory is bounded by the per-line cap (2 MiB), not by the unbounded chunk count. The per-line cap is checked before the unterminated byte buffer is extended (GW-15), so it holds even when no \n ever arrives, and the trailing-EOF path re-checks it before yielding. The gateway router also carries an explicit whole-body limit layer (GW-15: 2 MiB + 64 KiB framing headroom, answering 413) — a raw-Body handler never consults axum's DefaultBodyLimit (that limit is an extension extractors read), so the layer is the real backstop; the layer deliberately sits above the per-line cap so a single over-cap line still surfaces the route's semantic line-cap error rather than the plain-text 413. publish_schema validation is applied per chunk inside the sink-feeding stream (GW-01), so a Pub op registered with a publish_schema enforces the same per-chunk contract over HTTP as over the call protocol; a violation terminates the chunk stream with INVALID_INPUT (→ 422), the exact item shape an initiator-side call.error produces on the wire. A first line missing chunk (GW-10) is rejected INVALID_INPUT before dispatch — indistinguishable from a missing operation. The client disconnect abort path is unchanged: the dropped body stream propagates EOF through the framed reader into the sink's PublishStream.

Wire-shape note

On the call protocol, the initiator's chunks are call.published events over channel 0 or a stream; abort is call.aborted. Over HTTP, the abort path is the request being cut short: the client closing the connection early drops the body stream — the dispatch cancels the sink (the handler's PublishStream sees EOF, matching write-half close semantics).

to_openapi projection

The published gateway doc (ADR-045) describes /publish alongside the other five endpoints. The addition bumps the gateway contract's minor version. The per-caller operation surface remains discovered via /search (Pub ops are listed there); the doc does not preload operations.

What does not change

• to_mcp still exposes 4 tools and excludes both Sub and Pub (ADR-041; alkcall ADR-046) — MCP tool calls are request/response.
• The WS path needs no /publish equivalent: a WS session's channel 0 carries native call.published events.
• from_openapi/from_jsonschema produce no Pub operations in v1 (OpenAPI has no client-streaming representation).

Consequences

Positive:

• The gateway is a faithful projection of the call protocol's four operation types; no operation type is HTTP-unreachable.
• NDJSON is the natural HTTP encoding for a chunk stream (curl-able: echo '{"chunk":1}' | curl --data-binary @- -X POST .../publish).
• The dispatch path is invoke_sink() — the same spine as /call's invoke(), so the shared-dispatch invariants (identity, ACL, Internal filtering) hold by construction.
• The body is streamed line-by-line (memory bounded by the per-line cap; backpressure inherited from the HTTP body), so a client disconnect mid-stream cancels the sink exactly like a dropped write-half on the call protocol.

Negative:

• One more endpoint in the wire-stable gateway contract (version bump; one-way once published).
• HTTP has no in-band abort message: a mid-stream failure is indistinguishable from a network error to the server side (the handler sees EOF either way). Callers needing explicit failure semantics use the call protocol (WS channel 0).
• OQ-02 (first-line operation-naming convention) must settle before the /openapi.json version bumps — settled: first-line {operation, chunk} convention; terminal errors are plain HTTP status + JSON body (not an NDJSON line).
• The 2 MiB per-line cap bounds a single chunk (and, pre-newline, the unterminated buffer itself — GW-15); the total number of chunks is unbounded but the whole request body is capped by the gateway body-limit layer (2 MiB + 64 KiB headroom → 413, GW-15). Handlers that would receive unbounded streams over the wire get the same behavior over HTTP — operators front the endpoint with the same timeout controls used for any other streaming surface.

References

• http-server.md — the gateway dispatch, /publish section
• http-adapters.md — the gateway endpoint table
• ADR-042 — the gateway pattern this extends
• ADR-045 — version bump mechanics
• ADR-047 — the gateway as sole invoke path (now 6 endpoints)
• ADR-023 — error mapping
• alkcall ADR-046 (Publish Operation Type and HandlerKind::Sink) — the Pub/invoke_sink()/PublishStream machinery this endpoint exposes
• open-questions.md OQ-02 — first-line convention, error-envelope position