docs(plans): fuzzing plan for alkhttp (4 targets, sibling pattern)
Adopts the fuzzing pattern proven in alkcall/alktty/alktunnels/ alksocks: fuzz/ workspace with stable-toolchain shared invariant crate, committed seed corpora, corpus replay as the standing gate, detached runner for campaigns. Four targets planned — sse_parser and ws_chunk_framing (this session), spec_import and the stateful wave (later sessions). §3 table and §6 sequencing track progress.
This commit is contained in:
glm-5.3-flash committed
1 parent
40055170a0
commit
779e3ea908
1 file changed
+301
@@ -0,0 +1,301 @@
|
||||
# Plan: alkhttp fuzzing
|
||||
|
||||
Adopted from alkcall's `docs/research/fuzzing.md` (the pattern is
|
||||
operational in four sibling crates: alkcall, alktty, alktunnels,
|
||||
alksocks — each with the same `fuzz/` layout, the detached runner, and
|
||||
the corpus-replay-as-plain-test gate). This document is the working
|
||||
plan for alkhttp; the sibling docs are the reference for rationale
|
||||
(alkcall's doc is the deep version: tool landscape, comparable-crate
|
||||
survey, CI tiers) — this plan stays focused on what alkhttp fuzzes and
|
||||
in what order.
|
||||
|
||||
**Status:** in progress. Targets 1–2 (`sse_parser`,
|
||||
`ws_chunk_framing`) + infra are this session's scope; targets 3–4 are
|
||||
planned and can be handed to subsequent sessions.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why alkhttp fuzzes (the if)
|
||||
|
||||
Same three converging arguments as every sibling crate, with one
|
||||
stronger premise:
|
||||
|
||||
1. **This is the outermost shell.** alkcall's peers are authenticated
|
||||
by construction; alkhttp's HTTP surface faces anonymous callers —
|
||||
gateway routes accept bytes on the wire before any identity is
|
||||
resolved, `/healthz` and the stealth decoy are fully public, and the
|
||||
`from_*` adapters parse responses from remote HTTP servers nobody
|
||||
in this repo controls. The quinn-CVE class (RUSTSEC-2026-0037: a
|
||||
panic on attacker wire bytes is a remote DoS) applies here at its
|
||||
most direct. A remote crash in this crate does not need a peer
|
||||
trust step to reach.
|
||||
2. **Real hand-rolled byte parsing on untrusted streams.** The HTTP
|
||||
framing itself is hyper/axum/reqwest (upstream-fuzzed, not our job),
|
||||
but the crate owns an incremental SSE frame parser, the
|
||||
WS↔byte-stream chunk parser on both directions, a YAML→JSON
|
||||
conversion pass with its own number/key/pointer semantics, and
|
||||
bounded-schema compilation over arbitrary input values. These are
|
||||
the exact shapes (incremental, splitting-sensitive,
|
||||
cap-check-ordered) where example tests miss off-by-one bugs. No
|
||||
`unsafe` exists (grep-verified), so the expected yield is the
|
||||
alkcall profile: panics, arithmetic, allocation, and state-growth
|
||||
bugs — not memory corruption.
|
||||
3. **The integration seam is untested from this side.** alkhttp sits
|
||||
downstream of the alkcall 0.8.1 fix (duplicate adopt/open
|
||||
destroying the live channel — found by alkcall's `manager_routing`
|
||||
campaign minutes into its first run). alkhttp's WS path hands raw
|
||||
bytes to alkcall's demux; `fuzz/shared` depending on alkcall means
|
||||
corpus replay exercises the alkcall surface from the integration
|
||||
crate, a direction alkcall's own harness does not cover.
|
||||
|
||||
Also the standing empirical argument: the 0.8.1 bug was a stateful
|
||||
interleaving invisible to example-based tests, and it cost a patch
|
||||
release across every sibling. The infrastructure is already proven and
|
||||
cheap to replicate; the marginal cost here is only the target logic.
|
||||
|
||||
## 2. Infrastructure (identical to the siblings)
|
||||
|
||||
Layout (copy of alksocks/alktty/alktunnels/alkcall):
|
||||
|
||||
```
|
||||
fuzz/
|
||||
├── Cargo.toml alkhttp-fuzz (nightly-only bins; own [workspace])
|
||||
├── rust-toolchain.toml pins nightly + llvm-tools for this subtree only
|
||||
├── fuzz_targets/ thin fuzz_target! wrappers (3 lines each)
|
||||
├── shared/ alkhttp-fuzz-shared — STABLE-toolchain library:
|
||||
│ invariant logic + corpus-replay tests
|
||||
├── corpus/<target>/ committed seeds (generated by gen_fuzz_seeds.py)
|
||||
├── artifacts/ gitignored crash/oom/timeout artifacts + logs
|
||||
├── gen_fuzz_seeds.py deterministic seed generator (quiche pattern)
|
||||
├── json.dict JSON token dictionary (spec_import)
|
||||
├── run-detached.sh detached campaign runner (copied from the siblings)
|
||||
└── README.md operational cheat-sheet
|
||||
```
|
||||
|
||||
Load-bearing details (all four siblings hit these):
|
||||
|
||||
- **Invariant logic lives in `fuzz/shared/`**, not the target binaries.
|
||||
The stable-toolchain shared crate replays every committed seed
|
||||
through the identical invariant functions as plain `cargo test` —
|
||||
the standing fuzz gate (alkcall research §7.9 tier-3 deliverable,
|
||||
this repo has no hosted CI by policy) — and the `fuzz_target!`
|
||||
binaries are thin wrappers.
|
||||
- **Root `Cargo.toml` needs an explicit `[workspace]` table**
|
||||
(`members = ["."]`, `exclude = ["fuzz"]`); without it auto-discovery
|
||||
pulls `fuzz/shared/` into the main workspace and the stable
|
||||
toolchain builds nightly-consumed dev-deps. alkcall hit this trap.
|
||||
- **`fuzz/rust-toolchain.toml` pins nightly** so `cargo fuzz build`
|
||||
works from any CWD; nightly stays confined to `fuzz/`, MSRV 1.88
|
||||
untouched. `fuzz/` joins the publish `exclude` list.
|
||||
- **`.gitignore` additions**: `fuzz/artifacts/`, grown-corpus dirs
|
||||
(committed seeds stay).
|
||||
- **Detached runner (non-negotiable operating rule).** Campaigns never
|
||||
run as a foreground child of an agent session (an OOM in a target
|
||||
can take down the session host's process tree). Runner pins
|
||||
`-fork=1 -rss_limit_mb=2048 -malloc_limit_mb=2048 -timeout=25` and
|
||||
detaches via `setsid` + `nohup` + log redirect; the agent polls the
|
||||
log and artifact directory, never waits. Copied from alksocks
|
||||
(§`fuzz/run-detached.sh`).
|
||||
|
||||
**Verification-gate change:** `cargo test --manifest-path
|
||||
fuzz/shared/Cargo.toml` (corpus replay) joins AGENTS.md's verification
|
||||
checklist, as the siblings did.
|
||||
|
||||
## 3. Target inventory (all 4)
|
||||
|
||||
| # | Target | Drives | Input style | Status |
|
||||
|---|---|---|---|---|
|
||||
| 1 | `sse_parser` | `SseParser::feed` + `sse_event_envelope` mapping (`src/adapters/forward.rs`) | chunk-split sequencess × eof flag | implemented this session |
|
||||
| 2 | `ws_chunk_framing` | the WS↔byte-adapter chunk parsers (`run_write_pump` parse loop, inbound header path) (`src/websocket/byte_adapter.rs`) | byte sequences + message-split boundaries | implemented this session |
|
||||
| 3 | `spec_import` | `OpenAPISpec::from_str`/`from_yaml`/`from_json`, `FromJsonSchemaImporter`/`CompiledInputSchema`/`harden_closed_by_default`, `FromMCP::build_spec` (`src/adapters/openapi_spec.rs`, `from_jsonschema.rs`, `from_mcp/mod.rs`) | attacker-shaped OpenAPI/JSON/YAML docs + schemas | planned |
|
||||
| 4 | `ws_session_opseq` / `retry_throttle` | stateful op sequences over the WS byte-adapter pumps (write/read pump interleavings, close timing, oversize frames); `RetryAfterMiddleware` throttle-map growth | `#[derive(Arbitrary)]` op enums | planned (second wave) |
|
||||
|
||||
### Target 1 — `sse_parser` (the chunk_header-class target)
|
||||
|
||||
The crate's most direct untrusted-byte parser: any subscription over
|
||||
`from_openapi`/`from_jsonschema` pipes remote HTTP bytes through
|
||||
`SseParser::feed`.
|
||||
|
||||
- **Drives:** incremental `feed` with the fuzzer choosing the chunk
|
||||
split boundaries; also the full-input single feed, and the
|
||||
`sse_event_envelope` mapping (FWD-17: JSON payload surfaces as
|
||||
itself, non-JSON as the `{"data","event"}` wrapper, empty →
|
||||
`Null`).
|
||||
- **Invariants:**
|
||||
- no-panic on any byte sequence, any split, any eof timing;
|
||||
- splitting transparency: whole-feed and every-slice-split feed
|
||||
produce identical event streams (the alksocks `datagram_codec`
|
||||
pattern);
|
||||
- the buffer-cap check runs *before* the buffer grows
|
||||
(FWD-14): the reassembly buffer can never exceed
|
||||
`SSE_EVENT_BUFFER_CAP`; overflow surfaces as `BufferOverflow`,
|
||||
never a panic or unbounded growth;
|
||||
- dispatch discipline: an event is dispatched only on a blank line
|
||||
(or EOF for a pending event with data); `data:` lines join with
|
||||
`\n`; last `event:` before the blank line wins; `id:`/`retry:`/
|
||||
comment lines (`:` prefix) never dispatch or alter state;
|
||||
- shape partition: `BufferOverflow` only when a fed chunk would push
|
||||
the reassembly buffer over the cap; a cap-sized-and-under input
|
||||
can never trigger it — never allocate beyond the cap;
|
||||
- UTF-8 discipline: a multi-byte character split at a chunk boundary
|
||||
is not corrupted (decode happens once a complete line/EOF bounds
|
||||
the window); invalid raw UTF-8 lines are skipped silently (WHATWG
|
||||
tolerance), never panic;
|
||||
- envelope mapping (FWD-17): every `feed`-produced event maps to a
|
||||
success `ResponseEnvelope`; the JSON-ness branch agrees with
|
||||
`serde_json::from_str` on the joined data; the JSON branch never
|
||||
carries the `event` binding.
|
||||
- round-trip-ish probe: re-serializing a parsed event as
|
||||
`data:`/`event:` lines re-parses to an equal event for
|
||||
JSON-encodable payloads (catches field-name and joining drift;
|
||||
structural equality only).
|
||||
- **Exposure:** `SseParser`/`SseEvent`/`sse_event_envelope` are
|
||||
`pub(crate)`. Shared/invariant access via quinn/rustls-style
|
||||
exposure if the pure functions can't be reached — alkcall's
|
||||
campaign never needed `#[cfg(fuzzing)]` after making parse helpers
|
||||
`pub(crate)` + re-export; the first choice here is the same: a
|
||||
minimal `#[cfg(fuzzing)] pub mod fuzzing` re-export in
|
||||
`src/adapters/forward.rs` (nightly-fuzz-only builds see it; stable
|
||||
builds don't).
|
||||
|
||||
### Target 2 — `ws_chunk_framing`
|
||||
|
||||
The WS↔byte adapter parses 8-byte chunk headers on the write
|
||||
(outbound byte→WS-message) side and validates against
|
||||
`MAX_CHUNK_LEN`; the inbound side reassembles WS messages into the
|
||||
byte stream that alkcall's demux parses. The hand-rolled pieces are
|
||||
the chunk-walk loop and the pending accumulator.
|
||||
|
||||
- **Drives:** the write-pump parse loop driven directly with the
|
||||
same logic `run_write_pump` executes on each `WriteMsg::Bytes`
|
||||
(header scan → `MAX_CHUNK_LEN` gate → drain → 1 MiB message-piece
|
||||
split → pending cap), and the read-pump inbound path's
|
||||
message→header reassembly, driven as a byte-sequence target with
|
||||
the fuzzer controlling both the byte stream and the message split
|
||||
boundaries.
|
||||
- **Invariants:**
|
||||
- no-panic on any byte sequence at any message-split boundary;
|
||||
- a header claiming `> MAX_CHUNK_LEN` fails the stream loudly
|
||||
(WS-04/HY-09) at exactly the point the pump reaches the header —
|
||||
never a silent wait and never an allocate-before-reject;
|
||||
- the pending accumulator never exceeds `PENDING_BUFFER_CAP` =
|
||||
`MAX_CHUNK_LEN + 8` (WS-14): bound asserted after every op; a
|
||||
single write above the cap is rejected in `poll_write` before
|
||||
queueing, so the cap is invariantly satisfied;
|
||||
- message-splitting transparency: a chunk split across N
|
||||
WS-message feeds parses exactly like the whole-buffer feed
|
||||
(receiver's boundary is the header, not the message —
|
||||
WS-06); pieces split at any offset ≤ `WS_MESSAGE_CAP`
|
||||
reassemble losslessly;
|
||||
- consumption accounting: every parsed byte is either emitted as a
|
||||
chunk piece or retained in the pending buffer — nothing lost or
|
||||
duplicated across the walk;
|
||||
- the over-cap write rejection (`poll_write` > `PENDING_BUFFER_CAP`
|
||||
→ `InvalidData` before queueing) is exact: no partial state, no
|
||||
silent truncation.
|
||||
- **Exposure:** the pure parse helpers are extracted/testable; the
|
||||
pump functions are `pub(crate)` async — drives via a
|
||||
`#[cfg(fuzzing)]` re-export and a current-thread runtime with a
|
||||
mock sink (the alkcall `envelope_frame` Cursor pattern).
|
||||
|
||||
### Target 3 — `spec_import` (planned)
|
||||
|
||||
- **Drives:** raw bytes (and `json.dict`) → `serde_json`/`yaml_serde`
|
||||
→ `OpenAPISpec::from_str` (JSON-first, YAML-fallback, ADR-051) and
|
||||
the typed helpers: `yaml_to_json_value` boundary, number/key
|
||||
conversion; plus `CompiledInputSchema::compile` +
|
||||
`harden_closed_by_default` + `validate` over attacker-shaped
|
||||
schemas and input values; plus `FromMCP::build_spec` over arbitrary
|
||||
`Tool`-shaped JSON.
|
||||
- **Invariants:** no-panic; parse failures are always clean
|
||||
`AdapterError::SchemaParse` naming a pointer (never a panic/silent
|
||||
null); the YAML normalization rejects loudly what
|
||||
`serde_json::Number` cannot represent (non-finite floats,
|
||||
non-string keys); `$ref` resolution errors on cycles instead of
|
||||
unbounded recursion; the compiled validator rejects/accepts
|
||||
consistently w.r.t. the schema; closed-by-default hardening pins
|
||||
`additionalProperties: false`; registry-compile shape parity with
|
||||
alkcall's `spec_parse` invariants (bounded compile, always
|
||||
`Result`).
|
||||
- **Note:** yaml_serde (ADR-051: YAML 1.2 + merge keys) is the
|
||||
deepest third-party parse we own — the campaign should also encode
|
||||
the already-documented boundaries (`apply_merge` error path,
|
||||
duplicate-key rejection with line/column) as corpus expectations.
|
||||
|
||||
### Target 4 — stateful wave (planned)
|
||||
|
||||
- **`ws_session_opseq`:** `#[derive(Arbitrary)]` op sequences over
|
||||
the WS byte adapter driven end-to-end (like alktty's
|
||||
`session_opseq` / alkcall's `manager_routing`): write-side ops
|
||||
(`write(bytes)` ≤ cap, `shutdown`, oversized-write),
|
||||
read-side ops (feed messages, close, text message, oversize
|
||||
message), close timing both sides. Asserts: backpressure
|
||||
accounting (`READ_SLOTS`/`WRITE_SLOTS`), idle-timeout resets only
|
||||
on chunk progress (WS-01 semantics), close frames only at legal
|
||||
points, EOF sentinel semantics preserved, the alkcall 0.8.1 bug
|
||||
class (a rejected duplicate must leave live state intact) never
|
||||
reachable through this adapter.
|
||||
- **`retry_throttle`:** bounded-map growth of
|
||||
`RetryAfterMiddleware` (the inlined middleware): capacity bound,
|
||||
Farthest-future drop policy, ceiling clamp, expired-first
|
||||
eviction, jitter bounds; op sequences of responses with arbitrary
|
||||
`Retry-After` values (seconds, HTTP-date, garbage).
|
||||
- Sequenced after 1–3 stabilize, per the sibling pattern.
|
||||
|
||||
## 4. Corpus policy
|
||||
|
||||
Committed hand-made seeds per target, generated by
|
||||
`fuzz/gen_fuzz_seeds.py` (deterministic, in-tree, quiche pattern);
|
||||
grown corpora and artifacts gitignored. Seed menus:
|
||||
|
||||
- `sse_parser`: a minimal valid event; every framing edge (`data:`
|
||||
multi-line joins, `event:` override, comment, `id:`, `retry:`,
|
||||
BOM, CRLF/LF, field with no colon, `data` with no space);
|
||||
truncation at every structural prefix; split at a UTF-8 multibyte
|
||||
boundary; cap-boundary event (`SSE_EVENT_BUFFER_CAP - 1`,
|
||||
exactly at cap, one over); an over-cap trailing line; EOF with
|
||||
pending data / pending no-data; empty stream; pure whitespace.
|
||||
- `ws_chunk_framing`: a minimal valid chunk (`[id:u32 BE][len:u32
|
||||
BE][payload]`); `len = 0`, `len = MAX_CHUNK_LEN`,
|
||||
`len = MAX_CHUNK_LEN + 1`, `len = u32::MAX`; truncation at every
|
||||
prefix length (1–8 bytes); chunk split across a message boundary
|
||||
at every offset; channel-id 0 (the control channel); a header
|
||||
claiming big length whose payload never arrives (the pump-wait
|
||||
shape); the >cap single write (`poll_write` rejection).
|
||||
- `spec_import` (when built): a minimal valid OpenAPI doc (JSON);
|
||||
the same doc as YAML; missing `info`/`paths`; a deep-nesting
|
||||
probe; duplicate-key YAML; `<<: *anchor` merge; `.inf`/`.nan`
|
||||
scalars; non-string map keys; a `$ref` cycle; an unreachable
|
||||
`$ref`; `additionalProperties` absent (hardening probe) and
|
||||
present both ways.
|
||||
- Stateful seeds (target 4): the semantic fixtures the campaign must
|
||||
always replay — e.g. an adopted-then-teardown interleaving, a
|
||||
close-during-stream, a throttle-map cycle at capacity.
|
||||
|
||||
## 5. Campaign + gate policy
|
||||
|
||||
Identical to the siblings (alkcall §7.9 posture; no hosted CI in this
|
||||
repo):
|
||||
|
||||
- **Corpus replay is the standing fuzz gate:** `cargo test
|
||||
--manifest-path fuzz/shared/Cargo.toml` — in AGENTS.md's
|
||||
verification checklist.
|
||||
- **Campaigns run detached** via `fuzz/run-detached.sh` (never a
|
||||
foreground child of an agent session); budget 10 min per target for
|
||||
a smoke campaign, 30+ min before a release or after touching the
|
||||
SSE parser, the WS adapter, or the import adapters.
|
||||
- **Seeds regenerate deterministically:** `python3
|
||||
fuzz/gen_fuzz_seeds.py`.
|
||||
- Toolchain notes live in `fuzz/README.md`; nightly stays confined to
|
||||
`fuzz/`.
|
||||
|
||||
## 6. Sequencing
|
||||
|
||||
1. **This session** — infra (workspace exclude, toolchain pin, runner,
|
||||
seed generator, README) + targets 1–2 + corpora + corpus replay +
|
||||
AGENTS.md gate + smoke campaigns.
|
||||
2. **Next session(s)** — target 3 (`spec_import`) + campaigns.
|
||||
3. **Then** — target 4 stateful wave + longer campaigns; revisit
|
||||
corpus merges if any campaign yields inputs worth curating.
|
||||
|
||||
Status updates land in §3's table and §6's list as work completes.
|
||||
Reference in new issue
Block a user