82 lines
3.8 KiB
Markdown
82 lines
3.8 KiB
Markdown
---
|
|
id: review-002-gw15-publish-body-cap
|
|
name: Cap /publish body buffering pre-newline + explicit body-limit layer (GW-15)
|
|
status: completed
|
|
depends_on: []
|
|
scope: moderate
|
|
risk: medium
|
|
impact: component
|
|
level: implementation
|
|
tags: [gateway, review-002, security]
|
|
---
|
|
|
|
## Description
|
|
|
|
Review 002 GW-15 [major] — `/publish` lost both claimed memory bounds
|
|
in the GW-06 streaming rewrite. Verified at tree `91483a7`:
|
|
|
|
1. **No body limit on the route.** `publish_handler` takes raw
|
|
`axum::body::Body` (`src/gateway/routes.rs:249-253`). axum's
|
|
2 MiB `DefaultBodyLimit` is a request *extension* consulted only by
|
|
extractors (axum-core `request_parts.rs` returns the raw body
|
|
untouched) — a raw-Body handler has **no** default limit. The
|
|
module doc's claim ("in addition to axum's own 2 MiB default body
|
|
limit") is false on this route.
|
|
2. **Unbounded line buffer.** `BufferedLines::next_line`
|
|
(`routes.rs:425-448`) enforces `MAX_PUBLISH_LINE_BYTES` only when a
|
|
`\n` is found; bytes accumulate in `self.buffer` unboundedly until
|
|
then, and the trailing-EOF `std::mem::take(&mut self.buffer)` path
|
|
has no cap check at all.
|
|
|
|
Concrete failure: unauthenticated `POST /publish` with chunked
|
|
transfer-encoding streaming `'a'` forever (no newline) → heap grows
|
|
with the upload until OOM; each poll also re-scans the whole buffer
|
|
from index 0 → O(n²) CPU on top. This is the SRV-03 class one route
|
|
over (SRV-03 fixed `/mcp`; `/publish` regressed into the same class
|
|
via the streaming rewrite).
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] `BufferedLines` errors (terminal `LineError::LineCap` → the same
|
|
INVALID_INPUT family the line-cap error uses today) once
|
|
`self.buffer.len() > MAX_PUBLISH_LINE_BYTES`, checked *before*
|
|
extending, even with no newline seen — the trailing-EOF
|
|
`mem::take` path cannot exceed the cap
|
|
- [ ] The publish route (or the whole gateway router) carries an
|
|
explicit request-body-limit layer so it does not depend on
|
|
extractor-side defaults for raw-Body handlers (size = the 2 MiB
|
|
convention; note in module doc)
|
|
- [ ] Wire test: chunked upload with no `"\n"` for > cap bytes is
|
|
rejected with the documented status/body (not 200, not OOM)
|
|
- [ ] Wire test: EOF without newline after > cap buffered bytes also
|
|
rejected (the `mem::take` path)
|
|
- [ ] Existing publish tests still pass (first-line handling, NDJSON
|
|
streaming, schema validation, caps on terminated lines)
|
|
- [ ] `cargo test`, `cargo clippy --all-targets -- -D warnings`,
|
|
`cargo fmt --check` pass
|
|
|
|
## References
|
|
|
|
- docs/reviews/002-post-remediation-review.md (Part C', GW-15; Part G', COV-12 `BufferedLines` hot spot)
|
|
- docs/architecture/decisions/068-gateway-publish-endpoint.md
|
|
- src/gateway/routes.rs:249-253 (handler), 395-470 (BufferedLines)
|
|
- tasks/gateway/review-001-publish-schema-validation-robust.md (the cache this must not stale)
|
|
|
|
## Notes
|
|
|
|
The line-cap test that exists today (`publish_line_exceeding_cap…`)
|
|
short-circuits at the route level in single-chunk delivery — the new
|
|
pre-extend check must be exercised with a **streamed** (multi-chunk,
|
|
never-newline) body, which is the shape the old test cannot see. Keep
|
|
`MAX_PUBLISH_LINE_BYTES` as the single cap constant; a separate total
|
|
cap is optional (the layer limit bounds the total once landed).
|
|
|
|
GW-16 (the 400-vs-422 drift on the cap error's status) is deliberately
|
|
NOT in scope here — see review-002-gw16-status-drift; land the cap with
|
|
today's status first, then normalize statuses in that task to avoid
|
|
churn in the same lines.
|
|
|
|
## Summary
|
|
|
|
GATEWAY_BODY_LIMIT layer (2 MiB + 64 KiB headroom; Content-Length pre-rejection + counting stream, plain-text 413) + BufferedLines pre-extend cap on the residual tail; WIP's scan-before-extend bug fixed (complete lines now drain on arrival). 6 wire tests. ADR-068 updated; GW-16 status sites documented.
|