glm-5.3-flash
cc34c08e4e
Replace per-chunk parse_sse_frames with SseParser holding raw bytes across chunks: reassembles frames split at TCP boundaries (the review's silently-losing case), decodes UTF-8 per complete line so multi-byte chars split across chunks survive, caps the buffer at 1 MiB (SSE_EVENT_BUFFER_CAP) and dispatches a pending event at EOF. forward_stream threads the parser through its unfold state and emits a terminal error envelope on cap overflow. Existing single-chunk SSE test assertions preserved; added multi-chunk, split-UTF-8, EOF-dispatch, and cap tests. Verified: cargo test (219 pass), clippy -D warnings, fmt --check.
100 lines
4.3 KiB
Markdown
100 lines
4.3 KiB
Markdown
---
|
|
id: review-001-sse-parser
|
|
name: Incremental byte-level SSE parser (FWD-06)
|
|
status: completed
|
|
depends_on: []
|
|
scope: narrow
|
|
risk: high
|
|
impact: component
|
|
level: implementation
|
|
tags: [adapters, review-001]
|
|
---
|
|
|
|
## Description
|
|
|
|
Review 001 finding FWD-06 — silent subscription data loss, empirically
|
|
verified: `parse_sse_frames` (`src/adapters/forward.rs:409-417`) keeps
|
|
only the *last* line of each TCP chunk and discards any pending
|
|
multi-line `data_buffer`, so a chunk ending exactly at `data: …\n`
|
|
(blank line not yet arrived) silently loses the event. Verified: chunks
|
|
`"data: {\"n\":1}\n"` + `"\ndata: {\"n\":2}\n\n"` yield only event 2.
|
|
Single-chunk delivery (as in the tests) works, which is why the suite
|
|
passes.
|
|
|
|
Additional defects in the same parser: per-chunk `String::from_utf8_lossy`
|
|
(`:357`) corrupts multi-byte characters split at a chunk boundary (JSON
|
|
parse failure → event degraded to raw string); the trailing partial line
|
|
has no length cap (unbounded buffering); an event pending at EOF is
|
|
dropped (SSE says dispatch at EOF). For a subscription forwarder this is
|
|
silent data loss with no error signal.
|
|
|
|
Fix: an incremental **byte-level** parser carrying buffer state across
|
|
chunks (decode UTF-8 once over the reassembled buffer, not per chunk).
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [x] Multi-chunk test: event split across two TCP chunks is delivered (the review's empirically-verified case — the acceptance gate)
|
|
- [x] Split multi-byte UTF-8 across chunks parses (test)
|
|
- [x] Pending event dispatched at EOF; trailing partial line length-capped (tests)
|
|
- [x] Existing single-chunk SSE tests unchanged and green
|
|
- [x] `cargo test` and `cargo clippy --all-targets -- -D warnings` pass
|
|
|
|
## References
|
|
|
|
- docs/reviews/001-initial-implementation-review.md (Part D, FWD-06; Part I, COV-01)
|
|
|
|
## Notes
|
|
|
|
> Agent fills during implementation. Parser rewrite is isolated
|
|
> from the response-decode fixes (content-type, size caps, error bodies)
|
|
> in review-001-response-decoding so the delicate stateful rewrite lands
|
|
> alone. Same file — sequence or coordinate.
|
|
|
|
## Summary
|
|
|
|
> Filled on completion.
|
|
|
|
**Completed** — FWD-06 fixed via an incremental byte-level SSE parser.
|
|
|
|
### What changed
|
|
|
|
- `src/adapters/forward.rs`: replaced the per-chunk `parse_sse_frames`
|
|
function with `SseParser`, a stateful byte-level parser. It carries
|
|
the raw undecoded byte buffer across chunks, so a frame split at a
|
|
TCP boundary reassembles and a multi-byte UTF-8 character split at a
|
|
chunk boundary is no longer corrupted (UTF-8 is decoded per complete
|
|
line, after reassembly, not per chunk). Framing kept to what the call
|
|
protocol needs: lines split on `\n` with optional `\r`, `data:`
|
|
accumulation joined with `\n` on dispatch, `event:`/`id:`/`retry:`
|
|
accepted and ignored, comment lines skipped, blank line dispatches,
|
|
leading BOM stripped, pending event with data dispatched at EOF. A
|
|
documented `SSE_EVENT_BUFFER_CAP` (1 MiB) caps the reassembly buffer;
|
|
exceeding it yields `SseParseError::BufferOverflow`, which
|
|
`forward_stream` converts to a terminal error envelope instead of
|
|
buffering without bound.
|
|
- `forward_stream` now threads one `SseParser` through the
|
|
`stream::unfold` state (plus a `broken` flag so a parse/transport
|
|
error ends the stream) and dispatches the EOF-pending event when the
|
|
upstream byte stream ends.
|
|
- `src/adapters/from_openapi.rs`: the five existing single-chunk SSE
|
|
test assertions were preserved (same expected event shapes) and
|
|
ported to the new `feed` API; added the review's multi-chunk loss
|
|
case (`"data: {\"n\":1}\n"` + `"\ndata: {\"n\":2}\n\n"` → both
|
|
events), a split multi-byte UTF-8 test, an EOF-dispatch test, and an
|
|
oversized-partial-line cap test.
|
|
|
|
### Verification
|
|
|
|
- `cargo test` — 219 passed, 0 failed
|
|
- `cargo clippy --all-targets -- -D warnings` — clean
|
|
- `cargo fmt --check` — clean
|
|
|
|
### Notes for the sequential FWD-07/08/10/12 task
|
|
|
|
- The parser rejects nothing on malformed UTF-8 inside a well-formed
|
|
frame (invalid bytes in a `data:` value are ignored — no error); the
|
|
only error is the buffer cap. If FWD-10/12 adds error-body handling,
|
|
the `SseParseError` enum in forward.rs is the place to extend.
|
|
- `SseParser::feed` is synchronous and pure; it holds no I/O, so the
|
|
unfold-state shape `(bytes_stream, parser, broken)` can be reshaped
|
|
freely without touching parser logic. |