alkhttp/docs/plans/fuzzing.md

# Plan: alkhttp fuzzing

Adopted from alkcall's `docs/research/fuzzing.md` (the pattern is
operational in four sibling crates: alkcall, alktty, alktunnels,
alksocks — each with the same `fuzz/` layout, the detached runner, and
the corpus-replay-as-plain-test gate). This document is the working
plan for alkhttp; the sibling docs are the reference for rationale
(alkcall's doc is the deep version: tool landscape, comparable-crate
survey, CI tiers) — this plan stays focused on what alkhttp fuzzes and
in what order.

**Status:** in progress. Targets 1–2 (`sse_parser`,
`ws_chunk_framing`) + infra are this session's scope; targets 3–4 are
planned and can be handed to subsequent sessions.

---

## 1. Why alkhttp fuzzes (the if)

Same three converging arguments as every sibling crate, with one
stronger premise:

1. **This is the outermost shell.** alkcall's peers are authenticated
   by construction; alkhttp's HTTP surface faces anonymous callers —
   gateway routes accept bytes on the wire before any identity is
   resolved, `/healthz` and the stealth decoy are fully public, and the
   `from_*` adapters parse responses from remote HTTP servers nobody
   in this repo controls. The quinn-CVE class (RUSTSEC-2026-0037: a
   panic on attacker wire bytes is a remote DoS) applies here at its
   most direct. A remote crash in this crate does not need a peer
   trust step to reach.
2. **Real hand-rolled byte parsing on untrusted streams.** The HTTP
   framing itself is hyper/axum/reqwest (upstream-fuzzed, not our job),
   but the crate owns an incremental SSE frame parser, the
   WS↔byte-stream chunk parser on both directions, a YAML→JSON
   conversion pass with its own number/key/pointer semantics, and
   bounded-schema compilation over arbitrary input values. These are
   the exact shapes (incremental, splitting-sensitive,
   cap-check-ordered) where example tests miss off-by-one bugs. No
   `unsafe` exists (grep-verified), so the expected yield is the
   alkcall profile: panics, arithmetic, allocation, and state-growth
   bugs — not memory corruption.
3. **The integration seam is untested from this side.** alkhttp sits
   downstream of the alkcall 0.8.1 fix (duplicate adopt/open
   destroying the live channel — found by alkcall's `manager_routing`
   campaign minutes into its first run). alkhttp's WS path hands raw
   bytes to alkcall's demux; `fuzz/shared` depending on alkcall means
   corpus replay exercises the alkcall surface from the integration
   crate, a direction alkcall's own harness does not cover.

Also the standing empirical argument: the 0.8.1 bug was a stateful
interleaving invisible to example-based tests, and it cost a patch
release across every sibling. The infrastructure is already proven and
cheap to replicate; the marginal cost here is only the target logic.

## 2. Infrastructure (identical to the siblings)

Layout (copy of alksocks/alktty/alktunnels/alkcall):

```
fuzz/
├── Cargo.toml            alkhttp-fuzz (nightly-only bins; own [workspace])
├── rust-toolchain.toml   pins nightly + llvm-tools for this subtree only
├── fuzz_targets/         thin fuzz_target! wrappers (3 lines each)
├── shared/               alkhttp-fuzz-shared — STABLE-toolchain library:
│                         invariant logic + corpus-replay tests
├── corpus/<target>/      committed seeds (generated by gen_fuzz_seeds.py)
├── artifacts/            gitignored crash/oom/timeout artifacts + logs
├── gen_fuzz_seeds.py     deterministic seed generator (quiche pattern)
├── json.dict             JSON token dictionary (spec_import)
├── run-detached.sh       detached campaign runner (copied from the siblings)
└── README.md             operational cheat-sheet
```

Load-bearing details (all four siblings hit these):

- **Invariant logic lives in `fuzz/shared/`**, not the target binaries.
  The stable-toolchain shared crate replays every committed seed
  through the identical invariant functions as plain `cargo test` —
  the standing fuzz gate (alkcall research §7.9 tier-3 deliverable,
  this repo has no hosted CI by policy) — and the `fuzz_target!`
  binaries are thin wrappers.
- **Root `Cargo.toml` needs an explicit `[workspace]` table**
  (`members = ["."]`, `exclude = ["fuzz"]`); without it auto-discovery
  pulls `fuzz/shared/` into the main workspace and the stable
  toolchain builds nightly-consumed dev-deps. alkcall hit this trap.
- **`fuzz/rust-toolchain.toml` pins nightly** so `cargo fuzz build`
  works from any CWD; nightly stays confined to `fuzz/`, MSRV 1.88
  untouched. `fuzz/` joins the publish `exclude` list.
- **`.gitignore` additions**: `fuzz/artifacts/`, grown-corpus dirs
  (committed seeds stay).
- **Detached runner (non-negotiable operating rule).** Campaigns never
  run as a foreground child of an agent session (an OOM in a target
  can take down the session host's process tree). Runner pins
  `-fork=1 -rss_limit_mb=2048 -malloc_limit_mb=2048 -timeout=25` and
  detaches via `setsid` + `nohup` + log redirect; the agent polls the
  log and artifact directory, never waits. Copied from alksocks
  (§`fuzz/run-detached.sh`).

**Verification-gate change:** `cargo test --manifest-path
fuzz/shared/Cargo.toml` (corpus replay) joins AGENTS.md's verification
checklist, as the siblings did.

## 3. Target inventory (all 4)

| # | Target | Drives | Input style | Status |
|---|---|---|---|---|
| 1 | `sse_parser` | `SseParser::feed` + `sse_event_envelope` mapping (`src/adapters/forward.rs`) | chunk-split sequencess × eof flag | implemented this session |
| 2 | `ws_chunk_framing` | the WS↔byte-adapter chunk parsers (`run_write_pump` parse loop, inbound header path) (`src/websocket/byte_adapter.rs`) | byte sequences + message-split boundaries | implemented this session |
| 3 | `spec_import` | `OpenAPISpec::from_str`/`from_yaml`/`from_json`, `FromJsonSchemaImporter`/`CompiledInputSchema`/`harden_closed_by_default`, `FromMCP::build_spec` (`src/adapters/openapi_spec.rs`, `from_jsonschema.rs`, `from_mcp/mod.rs`) | attacker-shaped OpenAPI/JSON/YAML docs + schemas | planned |
| 4 | `ws_session_opseq` / `retry_throttle` | stateful op sequences over the WS byte-adapter pumps (write/read pump interleavings, close timing, oversize frames); `RetryAfterMiddleware` throttle-map growth | `#[derive(Arbitrary)]` op enums | planned (second wave) |

### Target 1 — `sse_parser` (the chunk_header-class target)

The crate's most direct untrusted-byte parser: any subscription over
`from_openapi`/`from_jsonschema` pipes remote HTTP bytes through
`SseParser::feed`.

- **Drives:** incremental `feed` with the fuzzer choosing the chunk
  split boundaries; also the full-input single feed, and the
  `sse_event_envelope` mapping (FWD-17: JSON payload surfaces as
  itself, non-JSON as the `{"data","event"}` wrapper, empty →
  `Null`).
- **Invariants:**
  - no-panic on any byte sequence, any split, any eof timing;
  - splitting transparency: whole-feed and every-slice-split feed
    produce identical event streams (the alksocks `datagram_codec`
    pattern);
  - the buffer-cap check runs *before* the buffer grows
    (FWD-14): the reassembly buffer can never exceed
    `SSE_EVENT_BUFFER_CAP`; overflow surfaces as `BufferOverflow`,
    never a panic or unbounded growth;
  - dispatch discipline: an event is dispatched only on a blank line
    (or EOF for a pending event with data); `data:` lines join with
    `\n`; last `event:` before the blank line wins; `id:`/`retry:`/
    comment lines (`:` prefix) never dispatch or alter state;
  - shape partition: `BufferOverflow` only when a fed chunk would push
    the reassembly buffer over the cap; a cap-sized-and-under input
    can never trigger it — never allocate beyond the cap;
  - UTF-8 discipline: a multi-byte character split at a chunk boundary
    is not corrupted (decode happens once a complete line/EOF bounds
    the window); invalid raw UTF-8 lines are skipped silently (WHATWG
    tolerance), never panic;
  - envelope mapping (FWD-17): every `feed`-produced event maps to a
    success `ResponseEnvelope`; the JSON-ness branch agrees with
    `serde_json::from_str` on the joined data; the JSON branch never
    carries the `event` binding.
  - round-trip-ish probe: re-serializing a parsed event as
    `data:`/`event:` lines re-parses to an equal event for
    JSON-encodable payloads (catches field-name and joining drift;
    structural equality only).
- **Exposure:** `SseParser`/`SseEvent`/`sse_event_envelope` are
  `pub(crate)`. Shared/invariant access via quinn/rustls-style
  exposure if the pure functions can't be reached — alkcall's
  campaign never needed `#[cfg(fuzzing)]` after making parse helpers
  `pub(crate)` + re-export; the first choice here is the same: a
  minimal `#[cfg(fuzzing)] pub mod fuzzing` re-export in
  `src/adapters/forward.rs` (nightly-fuzz-only builds see it; stable
  builds don't).

### Target 2 — `ws_chunk_framing`

The WS↔byte adapter parses 8-byte chunk headers on the write
(outbound byte→WS-message) side and validates against
`MAX_CHUNK_LEN`; the inbound side reassembles WS messages into the
byte stream that alkcall's demux parses. The hand-rolled pieces are
the chunk-walk loop and the pending accumulator.

- **Drives:** the write-pump parse loop driven directly with the
  same logic `run_write_pump` executes on each `WriteMsg::Bytes`
  (header scan → `MAX_CHUNK_LEN` gate → drain → 1 MiB message-piece
  split → pending cap), and the read-pump inbound path's
  message→header reassembly, driven as a byte-sequence target with
  the fuzzer controlling both the byte stream and the message split
  boundaries.
- **Invariants:**
  - no-panic on any byte sequence at any message-split boundary;
  - a header claiming `> MAX_CHUNK_LEN` fails the stream loudly
    (WS-04/HY-09) at exactly the point the pump reaches the header —
    never a silent wait and never an allocate-before-reject;
  - the pending accumulator never exceeds `PENDING_BUFFER_CAP` =
    `MAX_CHUNK_LEN + 8` (WS-14): bound asserted after every op; a
    single write above the cap is rejected in `poll_write` before
    queueing, so the cap is invariantly satisfied;
  - message-splitting transparency: a chunk split across N
    WS-message feeds parses exactly like the whole-buffer feed
    (receiver's boundary is the header, not the message —
    WS-06); pieces split at any offset ≤ `WS_MESSAGE_CAP`
    reassemble losslessly;
  - consumption accounting: every parsed byte is either emitted as a
    chunk piece or retained in the pending buffer — nothing lost or
    duplicated across the walk;
  - the over-cap write rejection (`poll_write` > `PENDING_BUFFER_CAP`
    → `InvalidData` before queueing) is exact: no partial state, no
    silent truncation.
- **Exposure:** the pure parse helpers are extracted/testable; the
  pump functions are `pub(crate)` async — drives via a
  `#[cfg(fuzzing)]` re-export and a current-thread runtime with a
  mock sink (the alkcall `envelope_frame` Cursor pattern).

### Target 3 — `spec_import` (planned)

- **Drives:** raw bytes (and `json.dict`) → `serde_json`/`yaml_serde`
  → `OpenAPISpec::from_str` (JSON-first, YAML-fallback, ADR-051) and
  the typed helpers: `yaml_to_json_value` boundary, number/key
  conversion; plus `CompiledInputSchema::compile` +
  `harden_closed_by_default` + `validate` over attacker-shaped
  schemas and input values; plus `FromMCP::build_spec` over arbitrary
  `Tool`-shaped JSON.
- **Invariants:** no-panic; parse failures are always clean
  `AdapterError::SchemaParse` naming a pointer (never a panic/silent
  null); the YAML normalization rejects loudly what
  `serde_json::Number` cannot represent (non-finite floats,
  non-string keys); `$ref` resolution errors on cycles instead of
  unbounded recursion; the compiled validator rejects/accepts
  consistently w.r.t. the schema; closed-by-default hardening pins
  `additionalProperties: false`; registry-compile shape parity with
  alkcall's `spec_parse` invariants (bounded compile, always
  `Result`).
- **Note:** yaml_serde (ADR-051: YAML 1.2 + merge keys) is the
  deepest third-party parse we own — the campaign should also encode
  the already-documented boundaries (`apply_merge` error path,
  duplicate-key rejection with line/column) as corpus expectations.

### Target 4 — stateful wave (planned)

- **`ws_session_opseq`:** `#[derive(Arbitrary)]` op sequences over
  the WS byte adapter driven end-to-end (like alktty's
  `session_opseq` / alkcall's `manager_routing`): write-side ops
  (`write(bytes)` ≤ cap, `shutdown`, oversized-write),
  read-side ops (feed messages, close, text message, oversize
  message), close timing both sides. Asserts: backpressure
  accounting (`READ_SLOTS`/`WRITE_SLOTS`), idle-timeout resets only
  on chunk progress (WS-01 semantics), close frames only at legal
  points, EOF sentinel semantics preserved, the alkcall 0.8.1 bug
  class (a rejected duplicate must leave live state intact) never
  reachable through this adapter.
- **`retry_throttle`:** bounded-map growth of
  `RetryAfterMiddleware` (the inlined middleware): capacity bound,
  Farthest-future drop policy, ceiling clamp, expired-first
  eviction, jitter bounds; op sequences of responses with arbitrary
  `Retry-After` values (seconds, HTTP-date, garbage).
- Sequenced after 1–3 stabilize, per the sibling pattern.

## 4. Corpus policy

Committed hand-made seeds per target, generated by
`fuzz/gen_fuzz_seeds.py` (deterministic, in-tree, quiche pattern);
grown corpora and artifacts gitignored. Seed menus:

- `sse_parser`: a minimal valid event; every framing edge (`data:`
  multi-line joins, `event:` override, comment, `id:`, `retry:`,
  BOM, CRLF/LF, field with no colon, `data` with no space);
  truncation at every structural prefix; split at a UTF-8 multibyte
  boundary; cap-boundary event (`SSE_EVENT_BUFFER_CAP - 1`,
  exactly at cap, one over); an over-cap trailing line; EOF with
  pending data / pending no-data; empty stream; pure whitespace.
- `ws_chunk_framing`: a minimal valid chunk (`[id:u32 BE][len:u32
  BE][payload]`); `len = 0`, `len = MAX_CHUNK_LEN`,
  `len = MAX_CHUNK_LEN + 1`, `len = u32::MAX`; truncation at every
  prefix length (1–8 bytes); chunk split across a message boundary
  at every offset; channel-id 0 (the control channel); a header
  claiming big length whose payload never arrives (the pump-wait
  shape); the >cap single write (`poll_write` rejection).
- `spec_import` (when built): a minimal valid OpenAPI doc (JSON);
  the same doc as YAML; missing `info`/`paths`; a deep-nesting
  probe; duplicate-key YAML; `<<: *anchor` merge; `.inf`/`.nan`
  scalars; non-string map keys; a `$ref` cycle; an unreachable
  `$ref`; `additionalProperties` absent (hardening probe) and
  present both ways.
- Stateful seeds (target 4): the semantic fixtures the campaign must
  always replay — e.g. an adopted-then-teardown interleaving, a
  close-during-stream, a throttle-map cycle at capacity.

## 5. Campaign + gate policy

Identical to the siblings (alkcall §7.9 posture; no hosted CI in this
repo):

- **Corpus replay is the standing fuzz gate:** `cargo test
  --manifest-path fuzz/shared/Cargo.toml` — in AGENTS.md's
  verification checklist.
- **Campaigns run detached** via `fuzz/run-detached.sh` (never a
  foreground child of an agent session); budget 10 min per target for
  a smoke campaign, 30+ min before a release or after touching the
  SSE parser, the WS adapter, or the import adapters.
- **Seeds regenerate deterministically:** `python3
  fuzz/gen_fuzz_seeds.py`.
- Toolchain notes live in `fuzz/README.md`; nightly stays confined to
  `fuzz/`.

## 6. Sequencing

1. **This session** — infra (workspace exclude, toolchain pin, runner,
   seed generator, README) + targets 1–2 + corpora + corpus replay +
   AGENTS.md gate + smoke campaigns.
2. **Next session(s)** — target 3 (`spec_import`) + campaigns.
3. **Then** — target 4 stateful wave + longer campaigns; revisit
   corpus merges if any campaign yields inputs worth curating.

Status updates land in §3's table and §6's list as work completes.