alkhttp/docs/plans/fuzzing.md

Plan: alkhttp fuzzing

Adopted from alkcall's docs/research/fuzzing.md (the pattern is operational in four sibling crates: alkcall, alktty, alktunnels, alksocks — each with the same fuzz/ layout, the detached runner, and the corpus-replay-as-plain-test gate). This document is the working plan for alkhttp; the sibling docs are the reference for rationale (alkcall's doc is the deep version: tool landscape, comparable-crate survey, CI tiers) — this plan stays focused on what alkhttp fuzzes and in what order.

Status: in progress. Targets 1–2 (sse_parser, ws_chunk_framing) + infra are this session's scope; targets 3–4 are planned and can be handed to subsequent sessions.

---

1. Why alkhttp fuzzes (the if)

Same three converging arguments as every sibling crate, with one stronger premise:

1. This is the outermost shell. alkcall's peers are authenticated by construction; alkhttp's HTTP surface faces anonymous callers — gateway routes accept bytes on the wire before any identity is resolved, /healthz and the stealth decoy are fully public, and the from_* adapters parse responses from remote HTTP servers nobody in this repo controls. The quinn-CVE class (RUSTSEC-2026-0037: a panic on attacker wire bytes is a remote DoS) applies here at its most direct. A remote crash in this crate does not need a peer trust step to reach.
2. Real hand-rolled byte parsing on untrusted streams. The HTTP framing itself is hyper/axum/reqwest (upstream-fuzzed, not our job), but the crate owns an incremental SSE frame parser, the WS↔byte-stream chunk parser on both directions, a YAML→JSON conversion pass with its own number/key/pointer semantics, and bounded-schema compilation over arbitrary input values. These are the exact shapes (incremental, splitting-sensitive, cap-check-ordered) where example tests miss off-by-one bugs. No unsafe exists (grep-verified), so the expected yield is the alkcall profile: panics, arithmetic, allocation, and state-growth bugs — not memory corruption.
3. The integration seam is untested from this side. alkhttp sits downstream of the alkcall 0.8.1 fix (duplicate adopt/open destroying the live channel — found by alkcall's manager_routing campaign minutes into its first run). alkhttp's WS path hands raw bytes to alkcall's demux; fuzz/shared depending on alkcall means corpus replay exercises the alkcall surface from the integration crate, a direction alkcall's own harness does not cover.

Also the standing empirical argument: the 0.8.1 bug was a stateful interleaving invisible to example-based tests, and it cost a patch release across every sibling. The infrastructure is already proven and cheap to replicate; the marginal cost here is only the target logic.

2. Infrastructure (identical to the siblings)

Layout (copy of alksocks/alktty/alktunnels/alkcall):

fuzz/
├── Cargo.toml            alkhttp-fuzz (nightly-only bins; own [workspace])
├── rust-toolchain.toml   pins nightly + llvm-tools for this subtree only
├── fuzz_targets/         thin fuzz_target! wrappers (3 lines each)
├── shared/               alkhttp-fuzz-shared — STABLE-toolchain library:
│                         invariant logic + corpus-replay tests
├── corpus/<target>/      committed seeds (generated by gen_fuzz_seeds.py)
├── artifacts/            gitignored crash/oom/timeout artifacts + logs
├── gen_fuzz_seeds.py     deterministic seed generator (quiche pattern)
├── json.dict             JSON token dictionary (spec_import)
├── run-detached.sh       detached campaign runner (copied from the siblings)
└── README.md             operational cheat-sheet

Load-bearing details (all four siblings hit these):

• Invariant logic lives in fuzz/shared/, not the target binaries. The stable-toolchain shared crate replays every committed seed through the identical invariant functions as plain cargo test — the standing fuzz gate (alkcall research §7.9 tier-3 deliverable, this repo has no hosted CI by policy) — and the fuzz_target! binaries are thin wrappers.
• Root Cargo.toml needs an explicit [workspace] table (members = ["."], exclude = ["fuzz"]); without it auto-discovery pulls fuzz/shared/ into the main workspace and the stable toolchain builds nightly-consumed dev-deps. alkcall hit this trap.
• fuzz/rust-toolchain.toml pins nightly so cargo fuzz build works from any CWD; nightly stays confined to fuzz/, MSRV 1.88 untouched. fuzz/ joins the publish exclude list.
• .gitignore additions: fuzz/artifacts/, grown-corpus dirs (committed seeds stay).
• Detached runner (non-negotiable operating rule). Campaigns never run as a foreground child of an agent session (an OOM in a target can take down the session host's process tree). Runner pins -fork=1 -rss_limit_mb=2048 -malloc_limit_mb=2048 -timeout=25 and detaches via setsid + nohup + log redirect; the agent polls the log and artifact directory, never waits. Copied from alksocks (§fuzz/run-detached.sh).

Verification-gate change: cargo test --manifest-path fuzz/shared/Cargo.toml (corpus replay) joins AGENTS.md's verification checklist, as the siblings did.

3. Target inventory (all 4)

#	Target	Drives	Input style	Status
1	sse_parser	SseParser::feed + sse_event_envelope mapping (src/adapters/forward.rs)	chunk-split sequencess × eof flag	implemented this session
2	ws_chunk_framing	the WS↔byte-adapter chunk parsers (run_write_pump parse loop, inbound header path) (src/websocket/byte_adapter.rs)	byte sequences + message-split boundaries	implemented this session
3	spec_import	OpenAPISpec::from_str/from_yaml/from_json, FromJsonSchemaImporter/CompiledInputSchema/harden_closed_by_default, FromMCP::build_spec (src/adapters/openapi_spec.rs, from_jsonschema.rs, from_mcp/mod.rs)	attacker-shaped OpenAPI/JSON/YAML docs + schemas	planned
4	ws_session_opseq / retry_throttle	stateful op sequences over the WS byte-adapter pumps (write/read pump interleavings, close timing, oversize frames); RetryAfterMiddleware throttle-map growth	#[derive(Arbitrary)] op enums	planned (second wave)

Target 1 — sse_parser (the chunk_header-class target)

The crate's most direct untrusted-byte parser: any subscription over from_openapi/from_jsonschema pipes remote HTTP bytes through SseParser::feed.

• Drives: incremental feed with the fuzzer choosing the chunk split boundaries; also the full-input single feed, and the sse_event_envelope mapping (FWD-17: JSON payload surfaces as itself, non-JSON as the {"data","event"} wrapper, empty → Null).
• Invariants:
  • no-panic on any byte sequence, any split, any eof timing;
  • splitting transparency: whole-feed and every-slice-split feed produce identical event streams (the alksocks datagram_codec pattern);
  • the buffer-cap check runs before the buffer grows (FWD-14): the reassembly buffer can never exceed SSE_EVENT_BUFFER_CAP; overflow surfaces as BufferOverflow, never a panic or unbounded growth;
  • dispatch discipline: an event is dispatched only on a blank line (or EOF for a pending event with data); data: lines join with \n; last event: before the blank line wins; id:/retry:/ comment lines (: prefix) never dispatch or alter state;
  • shape partition: BufferOverflow only when a fed chunk would push the reassembly buffer over the cap; a cap-sized-and-under input can never trigger it — never allocate beyond the cap;
  • UTF-8 discipline: a multi-byte character split at a chunk boundary is not corrupted (decode happens once a complete line/EOF bounds the window); invalid raw UTF-8 lines are skipped silently (WHATWG tolerance), never panic;
  • envelope mapping (FWD-17): every feed-produced event maps to a success ResponseEnvelope; the JSON-ness branch agrees with serde_json::from_str on the joined data; the JSON branch never carries the event binding.
  • round-trip-ish probe: re-serializing a parsed event as data:/event: lines re-parses to an equal event for JSON-encodable payloads (catches field-name and joining drift; structural equality only).
• Exposure: SseParser/SseEvent/sse_event_envelope are pub(crate). Shared/invariant access via quinn/rustls-style exposure if the pure functions can't be reached — alkcall's campaign never needed #[cfg(fuzzing)] after making parse helpers pub(crate) + re-export; the first choice here is the same: a minimal #[cfg(fuzzing)] pub mod fuzzing re-export in src/adapters/forward.rs (nightly-fuzz-only builds see it; stable builds don't).

Target 2 — ws_chunk_framing

The WS↔byte adapter parses 8-byte chunk headers on the write (outbound byte→WS-message) side and validates against MAX_CHUNK_LEN; the inbound side reassembles WS messages into the byte stream that alkcall's demux parses. The hand-rolled pieces are the chunk-walk loop and the pending accumulator.

• Drives: the write-pump parse loop driven directly with the same logic run_write_pump executes on each WriteMsg::Bytes (header scan → MAX_CHUNK_LEN gate → drain → 1 MiB message-piece split → pending cap), and the read-pump inbound path's message→header reassembly, driven as a byte-sequence target with the fuzzer controlling both the byte stream and the message split boundaries.
• Invariants:
  • no-panic on any byte sequence at any message-split boundary;
  • a header claiming > MAX_CHUNK_LEN fails the stream loudly (WS-04/HY-09) at exactly the point the pump reaches the header — never a silent wait and never an allocate-before-reject;
  • the pending accumulator never exceeds PENDING_BUFFER_CAP = MAX_CHUNK_LEN + 8 (WS-14): bound asserted after every op; a single write above the cap is rejected in poll_write before queueing, so the cap is invariantly satisfied;
  • message-splitting transparency: a chunk split across N WS-message feeds parses exactly like the whole-buffer feed (receiver's boundary is the header, not the message — WS-06); pieces split at any offset ≤ WS_MESSAGE_CAP reassemble losslessly;
  • consumption accounting: every parsed byte is either emitted as a chunk piece or retained in the pending buffer — nothing lost or duplicated across the walk;
  • the over-cap write rejection (poll_write > PENDING_BUFFER_CAP → InvalidData before queueing) is exact: no partial state, no silent truncation.
• Exposure: the pure parse helpers are extracted/testable; the pump functions are pub(crate) async — drives via a #[cfg(fuzzing)] re-export and a current-thread runtime with a mock sink (the alkcall envelope_frame Cursor pattern).

Target 3 — spec_import (planned)

• Drives: raw bytes (and json.dict) → serde_json/yaml_serde → OpenAPISpec::from_str (JSON-first, YAML-fallback, ADR-051) and the typed helpers: yaml_to_json_value boundary, number/key conversion; plus CompiledInputSchema::compile + harden_closed_by_default + validate over attacker-shaped schemas and input values; plus FromMCP::build_spec over arbitrary Tool-shaped JSON.
• Invariants: no-panic; parse failures are always clean AdapterError::SchemaParse naming a pointer (never a panic/silent null); the YAML normalization rejects loudly what serde_json::Number cannot represent (non-finite floats, non-string keys); $ref resolution errors on cycles instead of unbounded recursion; the compiled validator rejects/accepts consistently w.r.t. the schema; closed-by-default hardening pins additionalProperties: false; registry-compile shape parity with alkcall's spec_parse invariants (bounded compile, always Result).
• Note: yaml_serde (ADR-051: YAML 1.2 + merge keys) is the deepest third-party parse we own — the campaign should also encode the already-documented boundaries (apply_merge error path, duplicate-key rejection with line/column) as corpus expectations.

Target 4 — stateful wave (planned)

• ws_session_opseq: #[derive(Arbitrary)] op sequences over the WS byte adapter driven end-to-end (like alktty's session_opseq / alkcall's manager_routing): write-side ops (write(bytes) ≤ cap, shutdown, oversized-write), read-side ops (feed messages, close, text message, oversize message), close timing both sides. Asserts: backpressure accounting (READ_SLOTS/WRITE_SLOTS), idle-timeout resets only on chunk progress (WS-01 semantics), close frames only at legal points, EOF sentinel semantics preserved, the alkcall 0.8.1 bug class (a rejected duplicate must leave live state intact) never reachable through this adapter.
• retry_throttle: bounded-map growth of RetryAfterMiddleware (the inlined middleware): capacity bound, Farthest-future drop policy, ceiling clamp, expired-first eviction, jitter bounds; op sequences of responses with arbitrary Retry-After values (seconds, HTTP-date, garbage).
• Sequenced after 1–3 stabilize, per the sibling pattern.

4. Corpus policy

Committed hand-made seeds per target, generated by fuzz/gen_fuzz_seeds.py (deterministic, in-tree, quiche pattern); grown corpora and artifacts gitignored. Seed menus:

• sse_parser: a minimal valid event; every framing edge (data: multi-line joins, event: override, comment, id:, retry:, BOM, CRLF/LF, field with no colon, data with no space); truncation at every structural prefix; split at a UTF-8 multibyte boundary; cap-boundary event (SSE_EVENT_BUFFER_CAP - 1, exactly at cap, one over); an over-cap trailing line; EOF with pending data / pending no-data; empty stream; pure whitespace.
• ws_chunk_framing: a minimal valid chunk ([id:u32 BE][len:u32 BE][payload]); len = 0, len = MAX_CHUNK_LEN, len = MAX_CHUNK_LEN + 1, len = u32::MAX; truncation at every prefix length (1–8 bytes); chunk split across a message boundary at every offset; channel-id 0 (the control channel); a header claiming big length whose payload never arrives (the pump-wait shape); the >cap single write (poll_write rejection).
• spec_import (when built): a minimal valid OpenAPI doc (JSON); the same doc as YAML; missing info/paths; a deep-nesting probe; duplicate-key YAML; <<: *anchor merge; .inf/.nan scalars; non-string map keys; a $ref cycle; an unreachable $ref; additionalProperties absent (hardening probe) and present both ways.
• Stateful seeds (target 4): the semantic fixtures the campaign must always replay — e.g. an adopted-then-teardown interleaving, a close-during-stream, a throttle-map cycle at capacity.

5. Campaign + gate policy

Identical to the siblings (alkcall §7.9 posture; no hosted CI in this repo):

• Corpus replay is the standing fuzz gate: cargo test --manifest-path fuzz/shared/Cargo.toml — in AGENTS.md's verification checklist.
• Campaigns run detached via fuzz/run-detached.sh (never a foreground child of an agent session); budget 10 min per target for a smoke campaign, 30+ min before a release or after touching the SSE parser, the WS adapter, or the import adapters.
• Seeds regenerate deterministically: python3 fuzz/gen_fuzz_seeds.py.
• Toolchain notes live in fuzz/README.md; nightly stays confined to fuzz/.

6. Sequencing

1. This session — infra (workspace exclude, toolchain pin, runner, seed generator, README) + targets 1–2 + corpora + corpus replay + AGENTS.md gate + smoke campaigns.
2. Next session(s) — target 3 (spec_import) + campaigns.
3. Then — target 4 stateful wave + longer campaigns; revisit corpus merges if any campaign yields inputs worth curating.

Status updates land in §3's table and §6's list as work completes.