docs(tasks): decompose review-002 into 24 tasks (23 implementation + 1 bracketed follow-up)
Decomposition of docs/reviews/002-post-remediation-review.md per its 5-unit remediation plan: - Unit 1 (security-critical): gw15-publish-body-cap, prj16-schema-via-call (CF-004 filed alkcall-side), fwd13-dot-segments, fwd16-missing-capability, oai11-ref-memoization - Unit 2 (timeout/terminality): ws13-idle-progress, fwd15-stream-timeout, cli01-retry-after-budget, con17-mcp-pagination, con18-wss-sweep-exit - Unit 3 (projection/docs): projection-truthfulness, mcp-batch-cap, gw16-status-drift - Unit 4 (spec-import): yaml-normalization, oai13-path-item-wildcards, import-loudness-cluster, js01-placeholder-check, fwd17-19-contract-decisions - Unit 5 (WS polish + tests): con18b-ws-polish, client-policy-wire-tests, cov-deployment-knobs, cov13-dead-code, srv11-srv12-router-ordering - review-002-bracketed-followup: tentatively planned post-bulk pass (stale-check, OQA-18 enforcement decision, CON-08/09 close() lever, cross-crate re-checks) — deliberately not serialized against the bulk Also: review-002 numbering repair (CON-14 was double-booked; MCP pagination now CON-14, from_wss monitor renumbered CON-18, missing CON-14 section added). taskgraph: 66 valid, no cycles; 24 pending (all review-002); gen-1/gen-2 parallel waves identified; workflow-cost hotspots are prj16 (12.8) and ws13 (11.1), both carrying the reviewed slicing guidance in their Notes.
This commit is contained in:
@@ -0,0 +1,70 @@
|
||||
---
|
||||
id: review-002-cli01-retry-after-budget
|
||||
name: Budget-aware Retry-After sleep — stop re-arming a full ceiling per attempt (CLI-01)
|
||||
status: pending
|
||||
depends_on: []
|
||||
scope: narrow
|
||||
risk: medium
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [client, review-002]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Review 002 CLI-01 [major]. The `Retry-After` middleware sleeps outside
|
||||
the retry budget and re-arms a full ceiling on every throttled retry —
|
||||
middleware order is `RetryGateMiddleware` **outside**
|
||||
`RetryAfterMiddleware` (`http_client.rs:511-520`), so every retry
|
||||
attempt re-enters `RetryAfterMiddleware::handle`:
|
||||
|
||||
1. attempt 1 → `429` + `Retry-After: 300` → deadline recorded `t₀+300`
|
||||
2. reqwest-retry sleeps ≤ 2 s, retries
|
||||
3. attempt 2 re-sleeps until `t₀+300` (inside maybe_sleep_for), gets a
|
||||
fresh `429` → deadline **re-armed to `t₀+302`** (a full new 300 s)
|
||||
4. … for up to `max_retries` more → ~15 min wall time inside one
|
||||
`forward()` call
|
||||
|
||||
`TotalRetryBudget` gates only its own backoff sleeps; the 30 s request
|
||||
timeout never covers this window (the sleep happens before the reqwest
|
||||
request is constructed, so the timer never starts).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `maybe_sleep_for` is budget-aware: caps the sleep by the
|
||||
remaining `max_total_retry_duration` (check inside the sleep —
|
||||
the pending sleeps, clamped, or checks-and-skips when the budget
|
||||
is spent; implementer's choice, but a deadline that would extend
|
||||
past the budget must be truncated to it)
|
||||
- [ ] Re-arm clamp: `record_if_throttled` keeps the *earliest* deadline
|
||||
(or otherwise does not extend past the first-seen deadline for
|
||||
the same URL within one logical request) — a retry storm cannot
|
||||
push the wall clock out ceiling-per-attempt
|
||||
- [ ] Wire test: counting responder that always answers
|
||||
`429`/`Retry-After: N` with the test's ceiling set small → the
|
||||
caller's total wall time is bounded by max_total_retry_duration
|
||||
+ one attempt's request time (assert the bound, not the exact
|
||||
count)
|
||||
- [ ] The per-URL deadline map semantics for *separate* logical
|
||||
requests are unchanged (a fresh request still honors the
|
||||
recorded throttle window — that feature stays)
|
||||
- [ ] Fix the misspelled test name `malware_records_under_the_effective_url` while touching the file
|
||||
- [ ] `cargo test`, `cargo clippy --all-targets -- -D warnings`,
|
||||
`cargo fmt --check` pass
|
||||
|
||||
## References
|
||||
|
||||
- docs/reviews/002-post-remediation-review.md (Part D', CLI-01)
|
||||
- src/client/http_client.rs:511-520 (stack order), 370-395 (TotalRetryBudget), src/client/retry_after.rs:44-62 (ceiling), :127-211 (record/maybe_sleep)
|
||||
- tasks/client/review-001-client-timeout-retry.md (the Retry-After machinery)
|
||||
- tasks/client/review-002-client-policy-wire-tests.md (the counting-responder seam this test family shares)
|
||||
|
||||
## Notes
|
||||
|
||||
Keep the semantics decision narrow: the goal is "one logical request's
|
||||
Retry-After waits are bounded by the retry budget," NOT changing the
|
||||
cross-request throttle map (FWD-11's eviction/jitter behavior is
|
||||
already fixed and correct). The natural seam is threading the budget
|
||||
handle into the middleware order so RetryAfter sees it — small,
|
||||
testable, no public API change (the budget is already a
|
||||
`HttpClientConfig` field).
|
||||
@@ -0,0 +1,81 @@
|
||||
---
|
||||
id: review-002-client-policy-wire-tests
|
||||
name: Wire tests — redirect policy, retry gate/budget, streaming terminal arms, TLS error arms (CLI-02/03, COV-10/11)
|
||||
status: pending
|
||||
depends_on: [review-002-fwd15-stream-timeout, review-002-cli01-retry-after-budget]
|
||||
scope: moderate
|
||||
risk: low
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [client, review-002, coverage, adapters]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
The crate's highest-value coverage gap: the outbound client's policy
|
||||
stack — the code that protects credentials — has never run against a
|
||||
redirect or a retry on the wire, and `forward_stream`'s terminal arms
|
||||
are dark end-to-end (review 002 COV-11: `same_host_redirect_policy`
|
||||
100% dark; COV-10: SSE terminal arms dark). Concretely:
|
||||
|
||||
- **CLI-02 (redirect)**: no test in the crate sends any 3xx. Add:
|
||||
(a) same-host 302 → followed, credential headers forwarded;
|
||||
(b) cross-host 302 → response surfaced (HTTP_302-family), the second
|
||||
endpoint receives **zero requests** (count at the attacker server) —
|
||||
the FWD-03 property, pinned. Hop-cap (>10) → loud error, third test.
|
||||
The `spawn_responder` seam at forward.rs:1415-1477 already speaks
|
||||
raw HTTP and can emit `location:`.
|
||||
- **CLI-03 (retry)**: counting-responder tests: POST + 500 ⇒ exactly
|
||||
1 upstream hit (method gate, :362-366); GET + 500×2 ⇒ 3 hits then
|
||||
success; budget exhaustion (`max_total_retry_duration` small) stops
|
||||
retries even under a generous attempt cap (:381); PEM
|
||||
parse-failure arms (:487-507) via garbage-PEM files asserting
|
||||
`CaBundleParse`/`ClientCertParse` + path (never key material).
|
||||
- **COV-10 (streaming/forward terminal arms)**: (a) upstream SSE line
|
||||
> 1 MiB without newline through the real `forward_stream` → single
|
||||
INTERNAL terminal envelope, stream ends; (b) responder sends
|
||||
`data: a\n\n` then aborts the socket → terminal error envelope;
|
||||
(c) responder ends without trailing blank line → pending event
|
||||
dispatched at EOF; (d) dead-port transport failure through both
|
||||
`forward` (:719-722) and `forward_stream` (:851-853) → INTERNAL
|
||||
envelope; (e) binary `application/octet-stream` 200 → byte-array
|
||||
envelope (:638-643); (f) `application/json` 200 with malformed body
|
||||
→ INTERNAL decode envelope (:656-658).
|
||||
- Plus the small dark arms from COV-12 that live in this seam:
|
||||
ApiKey/Basic malformed-credential loud arms (extend the FWD-08 test
|
||||
family), declared-header-param invalid name/value rejections, and
|
||||
`SharedHttpClient::config()` reflecting a reloaded config
|
||||
(FWD-12's atomicity assertion).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Redirect: 3 tests (same-host follow, cross-host refuse +
|
||||
zero-contact assert, hop-cap error) passing
|
||||
- [ ] Retry: 3 tests (POST single-hit, GET retry-to-success with
|
||||
counted hits, budget-bounded) passing
|
||||
- [ ] TLS: 2 parse-failure tests (CaBundleParse, ClientCertParse) +
|
||||
config()-after-reload assertion
|
||||
- [ ] Streaming: 6 terminal-arm tests (oversized line, socket abort,
|
||||
EOF-flush, dead-port×2, binary decode, malformed-JSON decode)
|
||||
passing
|
||||
- [ ] Credential arms: ApiKey/Basic malformed-value tests land
|
||||
- [ ] `cargo test --all-features`, `cargo clippy --all-features
|
||||
--all-targets -- -D warnings`, `cargo fmt --check` pass
|
||||
|
||||
## References
|
||||
|
||||
- docs/reviews/002-post-remediation-review.md (Part D', CLI-02/03; Part G', COV-09/10/11/12)
|
||||
- src/client/http_client.rs:300-316 (redirect policy), :354-395 (retry gate/budget), :487-507 (PEM arms)
|
||||
- src/adapters/forward.rs:1415-1477 (spawn_responder seam), :887-925 (stream terminal arms)
|
||||
- tests/client_tls.rs (the TLS harness pattern)
|
||||
- tasks/client/review-002-fwd15-stream-timeout.md and review-002-cli01-retry-after-budget.md (land first — these tests pin their new behavior too)
|
||||
|
||||
## Notes
|
||||
|
||||
Pure test-addition task (plus any test-seam helpers). Land after the
|
||||
two client behavior tasks so the tests assert final semantics, not
|
||||
interim ones. Slice: (1) redirect tests, (2) retry tests, (3)
|
||||
streaming terminal arms, (4) TLS + config arms — independent commits.
|
||||
If `spawn_responder` needs a "count requests per host" surface, build
|
||||
it once and reuse — the cross-host zero-contact assertion is the
|
||||
load-bearing one.
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
id: review-002-fwd15-stream-timeout
|
||||
name: Subscriptions must not inherit the 30s total request timeout (FWD-15) + stream byte cap (FWD-14)
|
||||
status: pending
|
||||
depends_on: []
|
||||
scope: moderate
|
||||
risk: medium
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [client, review-002, adapters]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Review 002 FWD-15 [major] + FWD-14. Two streaming-path defects, same
|
||||
code region (`src/adapters/forward.rs:824-934`):
|
||||
|
||||
1. **FWD-15**: `forward_stream` sends through the shared client whose
|
||||
`DEFAULT_REQUEST_TIMEOUT` is 30 s — verified against reqwest 0.13.4
|
||||
that the total-timeout sleep rides into the response body stream.
|
||||
A healthy subscription dies at 30 s with `SSE stream error:
|
||||
operation timed out` — even keep-alive-emitting sources (the
|
||||
gateway deliberately runs subscriptions unbounded per ADR-021;
|
||||
dispatch.rs:28). The outbound half of the same subscription is
|
||||
mortally capped.
|
||||
2. **FWD-14**: the streaming branch has no size accounting — the 1 MiB
|
||||
cap bounds a single SSE *line*, not the stream. A hostile upstream
|
||||
emits well-formed 1-MiB-line events forever: 30 s of those is
|
||||
~GB/s into envelope allocation (mitigated only by the timeout —
|
||||
which FWD-15's fix then removes, so **both must land together or
|
||||
the fix makes FWD-14 worse**).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] FWD-15: `HandlerKind::Stream` forwards send with
|
||||
`request_timeout: None` while keeping connect + read timeouts
|
||||
(implementer's choice: per-request extension override verified
|
||||
against reqwest 0.13, or a second derived client built from the
|
||||
same config minus the total timeout) — a >30 s healthy
|
||||
subscription survives
|
||||
- [ ] FWD-15 wire test: responder trickling `: keepalive` comments past
|
||||
30 s (scaled: use a configurable short client timeout in the
|
||||
test) → stream still delivering events after the old deadline
|
||||
- [ ] FWD-14: a total byte cap on the streaming path (accumulate
|
||||
across `feed`; per-subscription total, default documented) and
|
||||
the line-cap check moved to *before* `extend_from_slice` so the
|
||||
overshoot cannot exceed cap + one chunk
|
||||
- [ ] FWD-14 test: a stream exceeding the total cap terminates with a
|
||||
single terminal error envelope (the stream-ends semantics)
|
||||
- [ ] Cap value + no-timeout decision documented in the module doc and
|
||||
http-server.md (ADR-049/021 note: unbounded *time* by design for
|
||||
subscriptions, bounded *bytes* per subscription)
|
||||
- [ ] `cargo test`, `cargo clippy --all-targets -- -D warnings`,
|
||||
`cargo fmt --check` pass
|
||||
|
||||
## References
|
||||
|
||||
- docs/reviews/002-post-remediation-review.md (Part D', FWD-14, FWD-15; Unit 2 rationale)
|
||||
- src/adapters/forward.rs:824-934 (send + streaming branch), :969-1082 (SSE parser), http_client.rs:69-95/:149-150/:164-178 (timeout config + read-timeout doc claim)
|
||||
- docs/architecture/decisions/049-streaming-handler-for-subscriptions.md
|
||||
- tasks/client/review-001-client-timeout-retry.md (the timeout defaults this revises)
|
||||
|
||||
## Notes
|
||||
|
||||
Order matters **within** this task: land FWD-15 with FWD-14 in the same
|
||||
commit-series — removing the total timeout without the byte cap opens
|
||||
an unbounded-memory window. The read-timeout (10/30 s stall guard)
|
||||
stays: it bounds upstream *staleness*; the doc at http_client.rs:149-150
|
||||
already correctly describes read-timeout as the stall guard (the total
|
||||
timeout's "stalled upstream" claim there is the thing being fixed).
|
||||
Coordinate with review-002-cli01-retry-after-budget (same file) and
|
||||
review-002-client-policy-wire-tests (which should follow this to test
|
||||
the new behavior) — sequence those.
|
||||
Reference in New Issue
Block a user