glm-5.3-flash
d7ee302046
- GW-01: /publish validates every NDJSON chunk against the op's publish_schema (incl. the first-line chunk) via NdjsonChunkStream — terminal Err(INVALID_INPUT)/422 on violation, matching the wire dispatcher's per-chunk contract. Route-level fix; the alkcall spine was explored and rejected (wire validation is pump-side by design). - GW-06: the body is streamed, not buffered — Body::into_data_stream() -> newline-framed BufferedLines -> lazily parsed chunk stream. ADR-068 documents the streamed semantics and the 2 MiB per-line cap. - GW-08: /batch capped at 100 operations (INVALID_INPUT 400). - GW-09: internal-op batch entries now carry generated UUID request ids. - GW-10: first publish line missing `chunk` is rejected INVALID_INPUT. - GW-11: redundant /publish pre-checks removed; enforcement rides on invoke_sink via the shared dispatch spine. - HY-13: the vacuous stub test was replaced by a body-cut-short test. - Adjacent: INVALID_OPERATION_TYPE now maps 422 (with identity) / 401 (without) in error.rs — the route relies on the shared mapper since the pre-checks are gone (GW-03's finding; was a 500 fall-through). Verification: cargo test 211 passed; cargo clippy --all-targets -- -D warnings clean; cargo fmt --check clean.
164 lines
7.5 KiB
Markdown
164 lines
7.5 KiB
Markdown
# ADR-068: Gateway `/publish` Endpoint for Pub Operations
|
|
|
|
## Status
|
|
|
|
Accepted
|
|
|
|
## Context
|
|
|
|
alkcall added `OperationType::Pub` and `HandlerKind::Sink`
|
|
(alkcall ADR-046): producer→consumer streaming, where the *initiator*
|
|
streams chunks to the operation via `call.published` events and the
|
|
handler consumes them as a `PublishStream`. This is the inverse of
|
|
`Sub` (consumer→producer streaming of results) and completes the four
|
|
operation types: `Query`, `Mutation`, `Sub`, `Pub`.
|
|
|
|
The HTTP gateway ([ADR-042](042-openapi-gateway-pattern.md),
|
|
[ADR-047](047-remove-direct-call-http-surface.md)) exposes five
|
|
endpoints: `/search`, `/schema`, `/call`, `/batch`, `/subscribe`. The
|
|
gateway's dispatch covers `Query`/`Mutation` (`/call`, `/batch`) and
|
|
`Sub` (`/subscribe` via the SSE projection,
|
|
[ADR-049](049-streaming-handler-for-subscriptions.md)). `Pub`
|
|
operations have no HTTP expression: an HTTP client cannot feed a
|
|
`Pub` operation's sink.
|
|
|
|
Without a surface, `Pub` operations are call-protocol-only (WS channel
|
|
0 or a QUIC/`alk/channels` session). That leaves HTTP clients —
|
|
curl, axios, a server-side script — unable to produce into `Pub` ops,
|
|
which breaks parity: every operation type reachable over the call
|
|
protocol should be reachable over HTTP, or the HTTP surface is not a
|
|
faithful projection.
|
|
|
|
## Decision
|
|
|
|
**The gateway gains a sixth endpoint: `POST /publish`.** It invokes a
|
|
`Pub` operation; the HTTP request body is the initiator's publish
|
|
stream; the operation's final `ResponseEnvelope` is the HTTP response.
|
|
|
|
### Request
|
|
|
|
- Path: `POST /publish`.
|
|
- Body: **newline-delimited JSON (NDJSON)** — each line is one
|
|
published chunk, serialized as a JSON value; the stream of lines maps
|
|
1:1 to `call.published` events. Chunk boundaries are the line
|
|
boundaries; a line's JSON value is the chunk payload.
|
|
- Auth: `Authorization: Bearer <token>` — same as every gateway
|
|
endpoint ([ADR-004](004-auth-as-shared-core.md)).
|
|
- The target operation is named the same way as `/call` — the body's
|
|
first line carries `{ "operation": "/{service}/{op}", "chunk": {...}`
|
|
for the first chunk, with subsequent lines carrying `chunk` values
|
|
only. (OQ-02 resolved: first-line convention, no query parameter, no
|
|
header; a `?operation=` parameter was rejected because it duplicates
|
|
the first-line field and complicates curl one-liners for no gain.)
|
|
|
|
### Dispatch
|
|
|
|
1. Resolve identity (Bearer → `resolve_from_token`).
|
|
2. Look up the operation; enforce `Visibility::External` (Internal →
|
|
`404`, same as `/call`) and `AccessControl::check` (→ `403`).
|
|
3. Verify `op_type == Pub` — a non-`Pub` op is
|
|
`INVALID_OPERATION_TYPE` → `400`.
|
|
4. Dispatch through `invoke_sink()` (alkcall ADR-046): stream each
|
|
NDJSON line as one `call.published` chunk into the handler's
|
|
`PublishStream`.
|
|
5. On end-of-body, deliver the handler's final `ResponseEnvelope`:
|
|
- `Ok(output)` → `200` with the output as JSON.
|
|
- `Err(call_error)` → mapped status per the standard error mapping
|
|
([ADR-023](023-operation-error-schemas.md), the gateway's
|
|
`HTTP_<status>` fidelity rules).
|
|
|
|
### Body handling (streamed, not buffered)
|
|
|
|
The NDJSON body is **streamed, never fully buffered** (GW-06): axum's
|
|
`Body` is framed into lines as bytes arrive, and each line is parsed and
|
|
pushed into the sink lazily — memory is bounded by the per-line cap (2
|
|
MiB, matching axum's default whole-body limit), not by the unbounded
|
|
chunk count. `publish_schema` validation is applied per chunk inside the
|
|
sink-feeding stream (GW-01), so a Pub op registered with a
|
|
`publish_schema` enforces the same per-chunk contract over HTTP as over
|
|
the call protocol; a violation terminates the chunk stream with
|
|
`INVALID_INPUT` (→ 422), the exact item shape an initiator-side
|
|
`call.error` produces on the wire. A first line missing `chunk` (GW-10)
|
|
is rejected `INVALID_INPUT` before dispatch — indistinguishable from a
|
|
missing `operation`. The client disconnect abort path is unchanged: the
|
|
dropped body stream propagates EOF through the framed reader into the
|
|
sink's `PublishStream`.
|
|
|
|
### Wire-shape note
|
|
|
|
On the call protocol, the initiator's chunks are `call.published`
|
|
events over channel 0 or a stream; abort is `call.aborted`. Over
|
|
HTTP, the abort path is the request being cut short: the client
|
|
closing the connection early drops the body stream — the dispatch
|
|
cancels the sink (the handler's `PublishStream` sees EOF, matching
|
|
write-half close semantics).
|
|
|
|
### `to_openapi` projection
|
|
|
|
The published gateway doc ([ADR-045](045-to-openapi-gateway-spec-versioning.md))
|
|
describes `/publish` alongside the other five endpoints. The addition
|
|
bumps the gateway contract's minor version. The per-caller operation
|
|
surface remains discovered via `/search` (`Pub` ops are listed there);
|
|
the doc does not preload operations.
|
|
|
|
### What does not change
|
|
|
|
- `to_mcp` still exposes 4 tools and excludes both `Sub` and `Pub`
|
|
([ADR-041](041-mcp-tool-gateway-pattern.md); alkcall ADR-046) — MCP
|
|
tool calls are request/response.
|
|
- The WS path needs no `/publish` equivalent: a WS session's channel 0
|
|
carries native `call.published` events.
|
|
- `from_openapi`/`from_jsonschema` produce no `Pub` operations in v1
|
|
(OpenAPI has no client-streaming representation).
|
|
|
|
## Consequences
|
|
|
|
**Positive:**
|
|
|
|
- The gateway is a faithful projection of the call protocol's four
|
|
operation types; no operation type is HTTP-unreachable.
|
|
- NDJSON is the natural HTTP encoding for a chunk stream (curl-able:
|
|
`echo '{"chunk":1}' | curl --data-binary @- -X POST .../publish`).
|
|
- The dispatch path is `invoke_sink()` — the same spine as `/call`'s
|
|
`invoke()`, so the shared-dispatch invariants (identity, ACL,
|
|
Internal filtering) hold by construction.
|
|
- The body is streamed line-by-line (memory bounded by the per-line
|
|
cap; backpressure inherited from the HTTP body), so a client
|
|
disconnect mid-stream cancels the sink exactly like a dropped
|
|
write-half on the call protocol.
|
|
|
|
**Negative:**
|
|
|
|
- One more endpoint in the wire-stable gateway contract (version bump;
|
|
one-way once published).
|
|
- HTTP has no in-band abort message: a mid-stream failure is
|
|
indistinguishable from a network error to the server side (the
|
|
handler sees EOF either way). Callers needing explicit failure
|
|
semantics use the call protocol (WS channel 0).
|
|
- ~~OQ-02 (first-line operation-naming convention) must settle before
|
|
the `/openapi.json` version bumps~~ — settled: first-line
|
|
`{operation, chunk}` convention; terminal errors are plain HTTP
|
|
status + JSON body (not an NDJSON line).
|
|
- The 2 MiB per-line cap (not a whole-body cap) bounds a single chunk;
|
|
the total number of chunks is unbounded. Handlers that would receive
|
|
unbounded streams over the wire get the same behavior over HTTP —
|
|
operators front the endpoint with the same body/timeout controls used
|
|
for any other streaming surface.
|
|
|
|
## References
|
|
|
|
- [http-server.md](../http-server.md) — the gateway dispatch, `/publish`
|
|
section
|
|
- [http-adapters.md](../http-adapters.md) — the gateway endpoint table
|
|
- [ADR-042](042-openapi-gateway-pattern.md) — the gateway pattern this
|
|
extends
|
|
- [ADR-045](045-to-openapi-gateway-spec-versioning.md) — version bump
|
|
mechanics
|
|
- [ADR-047](047-remove-direct-call-http-surface.md) — the gateway as
|
|
sole invoke path (now 6 endpoints)
|
|
- [ADR-023](023-operation-error-schemas.md) — error mapping
|
|
- alkcall ADR-046 (Publish Operation Type and `HandlerKind::Sink`) —
|
|
the `Pub`/`invoke_sink()`/`PublishStream` machinery this endpoint
|
|
exposes
|
|
- [open-questions.md](../open-questions.md) OQ-02 — first-line
|
|
convention, error-envelope position |