diff --git a/docs/plans/fuzzing.md b/docs/plans/fuzzing.md new file mode 100644 index 0000000..b18385f --- /dev/null +++ b/docs/plans/fuzzing.md @@ -0,0 +1,301 @@ +# Plan: alkhttp fuzzing + +Adopted from alkcall's `docs/research/fuzzing.md` (the pattern is +operational in four sibling crates: alkcall, alktty, alktunnels, +alksocks — each with the same `fuzz/` layout, the detached runner, and +the corpus-replay-as-plain-test gate). This document is the working +plan for alkhttp; the sibling docs are the reference for rationale +(alkcall's doc is the deep version: tool landscape, comparable-crate +survey, CI tiers) — this plan stays focused on what alkhttp fuzzes and +in what order. + +**Status:** in progress. Targets 1–2 (`sse_parser`, +`ws_chunk_framing`) + infra are this session's scope; targets 3–4 are +planned and can be handed to subsequent sessions. + +--- + +## 1. Why alkhttp fuzzes (the if) + +Same three converging arguments as every sibling crate, with one +stronger premise: + +1. **This is the outermost shell.** alkcall's peers are authenticated + by construction; alkhttp's HTTP surface faces anonymous callers — + gateway routes accept bytes on the wire before any identity is + resolved, `/healthz` and the stealth decoy are fully public, and the + `from_*` adapters parse responses from remote HTTP servers nobody + in this repo controls. The quinn-CVE class (RUSTSEC-2026-0037: a + panic on attacker wire bytes is a remote DoS) applies here at its + most direct. A remote crash in this crate does not need a peer + trust step to reach. +2. **Real hand-rolled byte parsing on untrusted streams.** The HTTP + framing itself is hyper/axum/reqwest (upstream-fuzzed, not our job), + but the crate owns an incremental SSE frame parser, the + WS↔byte-stream chunk parser on both directions, a YAML→JSON + conversion pass with its own number/key/pointer semantics, and + bounded-schema compilation over arbitrary input values. These are + the exact shapes (incremental, splitting-sensitive, + cap-check-ordered) where example tests miss off-by-one bugs. No + `unsafe` exists (grep-verified), so the expected yield is the + alkcall profile: panics, arithmetic, allocation, and state-growth + bugs — not memory corruption. +3. **The integration seam is untested from this side.** alkhttp sits + downstream of the alkcall 0.8.1 fix (duplicate adopt/open + destroying the live channel — found by alkcall's `manager_routing` + campaign minutes into its first run). alkhttp's WS path hands raw + bytes to alkcall's demux; `fuzz/shared` depending on alkcall means + corpus replay exercises the alkcall surface from the integration + crate, a direction alkcall's own harness does not cover. + +Also the standing empirical argument: the 0.8.1 bug was a stateful +interleaving invisible to example-based tests, and it cost a patch +release across every sibling. The infrastructure is already proven and +cheap to replicate; the marginal cost here is only the target logic. + +## 2. Infrastructure (identical to the siblings) + +Layout (copy of alksocks/alktty/alktunnels/alkcall): + +``` +fuzz/ +├── Cargo.toml alkhttp-fuzz (nightly-only bins; own [workspace]) +├── rust-toolchain.toml pins nightly + llvm-tools for this subtree only +├── fuzz_targets/ thin fuzz_target! wrappers (3 lines each) +├── shared/ alkhttp-fuzz-shared — STABLE-toolchain library: +│ invariant logic + corpus-replay tests +├── corpus// committed seeds (generated by gen_fuzz_seeds.py) +├── artifacts/ gitignored crash/oom/timeout artifacts + logs +├── gen_fuzz_seeds.py deterministic seed generator (quiche pattern) +├── json.dict JSON token dictionary (spec_import) +├── run-detached.sh detached campaign runner (copied from the siblings) +└── README.md operational cheat-sheet +``` + +Load-bearing details (all four siblings hit these): + +- **Invariant logic lives in `fuzz/shared/`**, not the target binaries. + The stable-toolchain shared crate replays every committed seed + through the identical invariant functions as plain `cargo test` — + the standing fuzz gate (alkcall research §7.9 tier-3 deliverable, + this repo has no hosted CI by policy) — and the `fuzz_target!` + binaries are thin wrappers. +- **Root `Cargo.toml` needs an explicit `[workspace]` table** + (`members = ["."]`, `exclude = ["fuzz"]`); without it auto-discovery + pulls `fuzz/shared/` into the main workspace and the stable + toolchain builds nightly-consumed dev-deps. alkcall hit this trap. +- **`fuzz/rust-toolchain.toml` pins nightly** so `cargo fuzz build` + works from any CWD; nightly stays confined to `fuzz/`, MSRV 1.88 + untouched. `fuzz/` joins the publish `exclude` list. +- **`.gitignore` additions**: `fuzz/artifacts/`, grown-corpus dirs + (committed seeds stay). +- **Detached runner (non-negotiable operating rule).** Campaigns never + run as a foreground child of an agent session (an OOM in a target + can take down the session host's process tree). Runner pins + `-fork=1 -rss_limit_mb=2048 -malloc_limit_mb=2048 -timeout=25` and + detaches via `setsid` + `nohup` + log redirect; the agent polls the + log and artifact directory, never waits. Copied from alksocks + (§`fuzz/run-detached.sh`). + +**Verification-gate change:** `cargo test --manifest-path +fuzz/shared/Cargo.toml` (corpus replay) joins AGENTS.md's verification +checklist, as the siblings did. + +## 3. Target inventory (all 4) + +| # | Target | Drives | Input style | Status | +|---|---|---|---|---| +| 1 | `sse_parser` | `SseParser::feed` + `sse_event_envelope` mapping (`src/adapters/forward.rs`) | chunk-split sequencess × eof flag | implemented this session | +| 2 | `ws_chunk_framing` | the WS↔byte-adapter chunk parsers (`run_write_pump` parse loop, inbound header path) (`src/websocket/byte_adapter.rs`) | byte sequences + message-split boundaries | implemented this session | +| 3 | `spec_import` | `OpenAPISpec::from_str`/`from_yaml`/`from_json`, `FromJsonSchemaImporter`/`CompiledInputSchema`/`harden_closed_by_default`, `FromMCP::build_spec` (`src/adapters/openapi_spec.rs`, `from_jsonschema.rs`, `from_mcp/mod.rs`) | attacker-shaped OpenAPI/JSON/YAML docs + schemas | planned | +| 4 | `ws_session_opseq` / `retry_throttle` | stateful op sequences over the WS byte-adapter pumps (write/read pump interleavings, close timing, oversize frames); `RetryAfterMiddleware` throttle-map growth | `#[derive(Arbitrary)]` op enums | planned (second wave) | + +### Target 1 — `sse_parser` (the chunk_header-class target) + +The crate's most direct untrusted-byte parser: any subscription over +`from_openapi`/`from_jsonschema` pipes remote HTTP bytes through +`SseParser::feed`. + +- **Drives:** incremental `feed` with the fuzzer choosing the chunk + split boundaries; also the full-input single feed, and the + `sse_event_envelope` mapping (FWD-17: JSON payload surfaces as + itself, non-JSON as the `{"data","event"}` wrapper, empty → + `Null`). +- **Invariants:** + - no-panic on any byte sequence, any split, any eof timing; + - splitting transparency: whole-feed and every-slice-split feed + produce identical event streams (the alksocks `datagram_codec` + pattern); + - the buffer-cap check runs *before* the buffer grows + (FWD-14): the reassembly buffer can never exceed + `SSE_EVENT_BUFFER_CAP`; overflow surfaces as `BufferOverflow`, + never a panic or unbounded growth; + - dispatch discipline: an event is dispatched only on a blank line + (or EOF for a pending event with data); `data:` lines join with + `\n`; last `event:` before the blank line wins; `id:`/`retry:`/ + comment lines (`:` prefix) never dispatch or alter state; + - shape partition: `BufferOverflow` only when a fed chunk would push + the reassembly buffer over the cap; a cap-sized-and-under input + can never trigger it — never allocate beyond the cap; + - UTF-8 discipline: a multi-byte character split at a chunk boundary + is not corrupted (decode happens once a complete line/EOF bounds + the window); invalid raw UTF-8 lines are skipped silently (WHATWG + tolerance), never panic; + - envelope mapping (FWD-17): every `feed`-produced event maps to a + success `ResponseEnvelope`; the JSON-ness branch agrees with + `serde_json::from_str` on the joined data; the JSON branch never + carries the `event` binding. + - round-trip-ish probe: re-serializing a parsed event as + `data:`/`event:` lines re-parses to an equal event for + JSON-encodable payloads (catches field-name and joining drift; + structural equality only). +- **Exposure:** `SseParser`/`SseEvent`/`sse_event_envelope` are + `pub(crate)`. Shared/invariant access via quinn/rustls-style + exposure if the pure functions can't be reached — alkcall's + campaign never needed `#[cfg(fuzzing)]` after making parse helpers + `pub(crate)` + re-export; the first choice here is the same: a + minimal `#[cfg(fuzzing)] pub mod fuzzing` re-export in + `src/adapters/forward.rs` (nightly-fuzz-only builds see it; stable + builds don't). + +### Target 2 — `ws_chunk_framing` + +The WS↔byte adapter parses 8-byte chunk headers on the write +(outbound byte→WS-message) side and validates against +`MAX_CHUNK_LEN`; the inbound side reassembles WS messages into the +byte stream that alkcall's demux parses. The hand-rolled pieces are +the chunk-walk loop and the pending accumulator. + +- **Drives:** the write-pump parse loop driven directly with the + same logic `run_write_pump` executes on each `WriteMsg::Bytes` + (header scan → `MAX_CHUNK_LEN` gate → drain → 1 MiB message-piece + split → pending cap), and the read-pump inbound path's + message→header reassembly, driven as a byte-sequence target with + the fuzzer controlling both the byte stream and the message split + boundaries. +- **Invariants:** + - no-panic on any byte sequence at any message-split boundary; + - a header claiming `> MAX_CHUNK_LEN` fails the stream loudly + (WS-04/HY-09) at exactly the point the pump reaches the header — + never a silent wait and never an allocate-before-reject; + - the pending accumulator never exceeds `PENDING_BUFFER_CAP` = + `MAX_CHUNK_LEN + 8` (WS-14): bound asserted after every op; a + single write above the cap is rejected in `poll_write` before + queueing, so the cap is invariantly satisfied; + - message-splitting transparency: a chunk split across N + WS-message feeds parses exactly like the whole-buffer feed + (receiver's boundary is the header, not the message — + WS-06); pieces split at any offset ≤ `WS_MESSAGE_CAP` + reassemble losslessly; + - consumption accounting: every parsed byte is either emitted as a + chunk piece or retained in the pending buffer — nothing lost or + duplicated across the walk; + - the over-cap write rejection (`poll_write` > `PENDING_BUFFER_CAP` + → `InvalidData` before queueing) is exact: no partial state, no + silent truncation. +- **Exposure:** the pure parse helpers are extracted/testable; the + pump functions are `pub(crate)` async — drives via a + `#[cfg(fuzzing)]` re-export and a current-thread runtime with a + mock sink (the alkcall `envelope_frame` Cursor pattern). + +### Target 3 — `spec_import` (planned) + +- **Drives:** raw bytes (and `json.dict`) → `serde_json`/`yaml_serde` + → `OpenAPISpec::from_str` (JSON-first, YAML-fallback, ADR-051) and + the typed helpers: `yaml_to_json_value` boundary, number/key + conversion; plus `CompiledInputSchema::compile` + + `harden_closed_by_default` + `validate` over attacker-shaped + schemas and input values; plus `FromMCP::build_spec` over arbitrary + `Tool`-shaped JSON. +- **Invariants:** no-panic; parse failures are always clean + `AdapterError::SchemaParse` naming a pointer (never a panic/silent + null); the YAML normalization rejects loudly what + `serde_json::Number` cannot represent (non-finite floats, + non-string keys); `$ref` resolution errors on cycles instead of + unbounded recursion; the compiled validator rejects/accepts + consistently w.r.t. the schema; closed-by-default hardening pins + `additionalProperties: false`; registry-compile shape parity with + alkcall's `spec_parse` invariants (bounded compile, always + `Result`). +- **Note:** yaml_serde (ADR-051: YAML 1.2 + merge keys) is the + deepest third-party parse we own — the campaign should also encode + the already-documented boundaries (`apply_merge` error path, + duplicate-key rejection with line/column) as corpus expectations. + +### Target 4 — stateful wave (planned) + +- **`ws_session_opseq`:** `#[derive(Arbitrary)]` op sequences over + the WS byte adapter driven end-to-end (like alktty's + `session_opseq` / alkcall's `manager_routing`): write-side ops + (`write(bytes)` ≤ cap, `shutdown`, oversized-write), + read-side ops (feed messages, close, text message, oversize + message), close timing both sides. Asserts: backpressure + accounting (`READ_SLOTS`/`WRITE_SLOTS`), idle-timeout resets only + on chunk progress (WS-01 semantics), close frames only at legal + points, EOF sentinel semantics preserved, the alkcall 0.8.1 bug + class (a rejected duplicate must leave live state intact) never + reachable through this adapter. +- **`retry_throttle`:** bounded-map growth of + `RetryAfterMiddleware` (the inlined middleware): capacity bound, + Farthest-future drop policy, ceiling clamp, expired-first + eviction, jitter bounds; op sequences of responses with arbitrary + `Retry-After` values (seconds, HTTP-date, garbage). +- Sequenced after 1–3 stabilize, per the sibling pattern. + +## 4. Corpus policy + +Committed hand-made seeds per target, generated by +`fuzz/gen_fuzz_seeds.py` (deterministic, in-tree, quiche pattern); +grown corpora and artifacts gitignored. Seed menus: + +- `sse_parser`: a minimal valid event; every framing edge (`data:` + multi-line joins, `event:` override, comment, `id:`, `retry:`, + BOM, CRLF/LF, field with no colon, `data` with no space); + truncation at every structural prefix; split at a UTF-8 multibyte + boundary; cap-boundary event (`SSE_EVENT_BUFFER_CAP - 1`, + exactly at cap, one over); an over-cap trailing line; EOF with + pending data / pending no-data; empty stream; pure whitespace. +- `ws_chunk_framing`: a minimal valid chunk (`[id:u32 BE][len:u32 + BE][payload]`); `len = 0`, `len = MAX_CHUNK_LEN`, + `len = MAX_CHUNK_LEN + 1`, `len = u32::MAX`; truncation at every + prefix length (1–8 bytes); chunk split across a message boundary + at every offset; channel-id 0 (the control channel); a header + claiming big length whose payload never arrives (the pump-wait + shape); the >cap single write (`poll_write` rejection). +- `spec_import` (when built): a minimal valid OpenAPI doc (JSON); + the same doc as YAML; missing `info`/`paths`; a deep-nesting + probe; duplicate-key YAML; `<<: *anchor` merge; `.inf`/`.nan` + scalars; non-string map keys; a `$ref` cycle; an unreachable + `$ref`; `additionalProperties` absent (hardening probe) and + present both ways. +- Stateful seeds (target 4): the semantic fixtures the campaign must + always replay — e.g. an adopted-then-teardown interleaving, a + close-during-stream, a throttle-map cycle at capacity. + +## 5. Campaign + gate policy + +Identical to the siblings (alkcall §7.9 posture; no hosted CI in this +repo): + +- **Corpus replay is the standing fuzz gate:** `cargo test + --manifest-path fuzz/shared/Cargo.toml` — in AGENTS.md's + verification checklist. +- **Campaigns run detached** via `fuzz/run-detached.sh` (never a + foreground child of an agent session); budget 10 min per target for + a smoke campaign, 30+ min before a release or after touching the + SSE parser, the WS adapter, or the import adapters. +- **Seeds regenerate deterministically:** `python3 + fuzz/gen_fuzz_seeds.py`. +- Toolchain notes live in `fuzz/README.md`; nightly stays confined to + `fuzz/`. + +## 6. Sequencing + +1. **This session** — infra (workspace exclude, toolchain pin, runner, + seed generator, README) + targets 1–2 + corpora + corpus replay + + AGENTS.md gate + smoke campaigns. +2. **Next session(s)** — target 3 (`spec_import`) + campaigns. +3. **Then** — target 4 stateful wave + longer campaigns; revisit + corpus merges if any campaign yields inputs worth curating. + +Status updates land in §3's table and §6's list as work completes. \ No newline at end of file