diff --git a/docs/plans/fuzzing.md b/docs/plans/fuzzing.md new file mode 100644 index 0000000..5afa910 --- /dev/null +++ b/docs/plans/fuzzing.md @@ -0,0 +1,407 @@ +# Plan: alktype fuzzing + +Adopted from alkhttp's `docs/plans/fuzzing.md` (the pattern is operational +in six sibling crates: alkcall, alktty, alktunnels, alksocks, alkhttp — +each with the same `fuzz/` layout, the detached runner, and the +corpus-replay-as-plain-test gate). The rationale research lives in +alkcall's `docs/research/fuzzing.md` (tool landscape, comparable-crate +survey, the no-hosted-CI policy §7.9); this plan stays focused on what +alktype fuzzes and in what order. + +Rationale for this crate in one paragraph (the detailed version applies +by reference from the two docs above): alktype is the binary engine the +alk* family consumes — alkcall's hub/spoke accepts BAST schema documents +from arbitrary internet peers, and those documents flow into this +crate's compile paths downstream; both untrusted-input shapes exist here +(attacker-shaped JSON BAST docs → `AlkTypeEngine::compile`, and +attacker-shaped byte buffers read according to a schema → +`validate_bytes` / `SequentialReader` / `tunion` dispatch / +`materialize`), and the byte side is hand-rolled decode +(`data_access`, `tunion` discriminators, indirect `{offset,length}` +pairs) — exactly the shapes where example tests miss off-by-one bugs. +The crate is fully synchronous, so targets are simpler than +alkcall/alkhttp's (no current-thread runtime shims anywhere). + +**Status:** plan adopted (2026-09-30); implementation not started. The +pre-fuzzing inventory (§6) is verified against the code at 0.3.0. No +`fuzz/` directory exists and no finding has been logged. + +--- + +## 1. Why alktype fuzzes (the if) + +1. **Downstream of the trust boundary.** alktype is compiled against in + alkcall (the integration crate), whose peers are untrusted and whose + wire payloads carry schema-shaped JSON. A panic on a malicious BAST + doc or bytes read under one is the quinn-CVE class + (RUSTSEC-2026-0037) at one further hop: the alk* stack parses + documents it never vetted, and alktype is where they get walked. +2. **Both input shapes, one crate.** Sibling crates each had mostly one + parse shape (wire bytes); alktype has the schema-JSON shape *and* + the raw-buffer shape, plus two-input combined paths + (`read_field`/`write_field`, `materialize_aligned` are doc+bytes). +3. **Infrastructure is proven and cheap; the crate is the simplest + consumer yet.** Six siblings run the layout; alktype is sync, has + zero `unsafe`, zero `unwrap`/`expect` outside tests, and no + allocation-from-wire-count anywhere (grep-verified inventory). The + marginal cost is target logic only. + +**Honest caveat (alkcall §1's shape):** the code is already well +hardened — `checked_add`/`check_bounds` everywhere, parse-time caps +(`MAX_ARRAY_ELEMENTS`, `MAX_ARRAY_BYTES`, `MAX_ALIGN` = 4096, +`MAX_LENGTH`), `MAX_GRAPH_DEPTH`/`MAX_COMPILE_DEPTH` = 128, meta-schema +gate before any walker. Expected yield is low-moderate: the residual +candidates in §6 are the first things to probe; a clean first campaign +is the successful negative result — "we think the engine is robust" +converted into a demonstrated property. + +## 2. Infrastructure (identical to the siblings) + +Layout (copy of alkcall/alkhttp): + +``` +fuzz/ +├── Cargo.toml alktype-fuzz (nightly-only bins; own [workspace]) +├── rust-toolchain.toml pins nightly + llvm-tools for this subtree only +├── fuzz_targets/ thin fuzz_target! wrappers (3 lines each) +├── shared/ alktype-fuzz-shared — STABLE-toolchain library: +│ invariant logic + corpus-replay tests +├── corpus// committed seeds (generated by gen_fuzz_seeds.py) +├── artifacts/ gitignored crash/oom/timeout artifacts + logs +├── gen_fuzz_seeds.py deterministic seed generator (quiche pattern) +├── json.dict JSON/BAST token dictionary (bast_compile) +├── run-detached.sh detached campaign runner (copied from the siblings) +└── README.md operational cheat-sheet +``` + +Load-bearing details (all six siblings hit these; alkcall's doc is the +deep reference): + +- **Invariant logic lives in `fuzz/shared/`**, not the target binaries. + The stable-toolchain shared crate replays every committed seed through + the identical invariant functions as plain `cargo test` — the standing + fuzz gate (alkcall §7.9 tier-3 deliverable; no hosted CI in this repo + by policy). The `fuzz_target!` binaries are thin wrappers. +- **Root `Cargo.toml` needs an explicit `[workspace]` table** + (`members = ["."]`, `exclude = ["fuzz"]`); without it auto-discovery + pulls `fuzz/shared/` into the main workspace and the stable toolchain + builds nightly-consumed dev-deps. alkcall hit this trap. +- **`fuzz/rust-toolchain.toml` pins nightly** so `cargo fuzz build` + works from any CWD; nightly stays confined to `fuzz/`, MSRV 1.85 + untouched here. `fuzz/` joins the publish `exclude` list. +- **`.gitignore` additions**: `fuzz/artifacts/`, grown-corpus dirs + (committed seeds stay). +- **Detached runner (non-negotiable operating rule).** Campaigns never + run as a foreground child of an agent session; the runner pins + `-fork=1 -rss_limit_mb=2048 -malloc_limit_mb=2048 -timeout=25` and + detaches via `setsid` + `nohup` + log redirect; the agent polls the + log and artifact directory, never waits. Copied from the siblings. +- **No feature-gating needed in `fuzz/shared`**: alktype has + `default = []` and no feature flags, so the shared crate rides the + main crate build unconditionally (unlike alkhttp's gated + `openapi`/`mcp`). +- **Exposure needs are minimal.** The inventory found every target + entry point already `pub` (`compile`, `data_access::*`, + `SequentialReader`, `LayoutBuilder`, `tunion::*`, `materialize::*`, + `validate_bast_doc`, `build_validator`). No `#[cfg(fuzzing)]` hub is + expected — the first choice remains a minimal `fuzzing` hub only if a + needed item turns out `pub(crate)`, per the sibling pattern (alkcall + never needed one). + +**Verification-gate change:** `cargo test --manifest-path +fuzz/shared/Cargo.toml` (corpus replay) joins AGENTS.md's verification +checklist, as the siblings did. + +## 3. Target inventory (5, in waves) + +| # | Target | Drives | Input style | Status | +|---|---|---|---|---| +| 1 | `bast_compile` | `AlkTypeEngine::compile` both modes (via `bast_meta` → `BastDoc` → plans → layout → validator) | raw bytes → serde_json → BAST doc | planned | +| 2 | `data_access` | the hand-rolled decode core (`src/data_access.rs`, read side) | raw `&[u8]` + chosen (offset, endian) | planned | +| 3 | `read_opseq` | stateful `SequentialReader` op sequences over hostile bytes under a fixed plan | `#[derive(Arbitrary)]` op enum | planned (wave 2) | +| 4 | `layout_build` | `LayoutBuilder::build` with adversarial `var_sizes` | `#[derive(Arbitrary)]` map shapes | planned (wave 2) | +| 5 | `validate_pair` | two-input structured: compile a schema once per exec, hammer hostile bytes through `validate_bytes`/`read_field`/`materialize` | `#[derive(Arbitrary)]` (doc, bytes) pair | planned (wave 3) | + +### Target 1 — `bast_compile` (the whole schema side, one choke point) + +`AlkTypeEngine::compile` (`src/engine.rs:127-176`) fans out through the +entire untrusted-JSON surface: `bast_meta::validate_bast_doc` → +`BastDoc::new` → `ValidationPlan::compile` → `LayoutBuilder::new` + +`ReadPlan::compile` (packed) or `OffsetMap::compute` (aligned) → +`validation::build_validator` (when `json_schema` is `Some`). + +- **Drives:** raw bytes → `serde_json` → `compile(value, root_name, + mode, None)` in both modes; a second lane feeds `Some(schema)` with a + second attacker-shaped JSON value for the jsonschema-build path. +- **Invariants:** + - no-panic on any JSON document, both modes; + - compile is always `Result` — every rejection is a clean + `AlkTypeError` (`Schema`/`Offset`/`Validation` payload classes, + `src/error.rs:11-28`), never a panic or a silent bogus engine; + - meta-schema gate ordering: any doc that fails + `validate_bast_doc` must surface `Schema(...)` and must never reach + layout/plan walks (shape partition); + - parse-time caps hold exactly: `align > 4096`, arrays above + `MAX_ARRAY_ELEMENTS`/`MAX_ARRAY_BYTES`, `maxLength > + MAX_LENGTH`, depth > 128, and `$ref` cycles all reject at compile + with the documented error classes (the walk-guard + `check_ref_graph`, compile-depth, and cycle-`seen` machinery + pinned by adversarial corpus entries); + - if compile fails in packed it must also fail in aligned (mode + independence of the schema-gate layer — the parse layers are + shared; divergence means a mode-specific parse bug); + - a successfully compiled engine's `endian()` equals the root + struct's declared endianness. +- **Seeds:** the full BAST feature menu (each kind, endian ×2, TUnion + byte/field/enum discriminators, records, arrays, string/bytes + encodings, `$ref` diamond), each reject-class corpus entry, plus the + hostile menu in §4. + +### Target 2 — `data_access` (the decode core) + +Every byte-touching decode funnels through `read_array` +(`src/data_access.rs:48-75`): `checked_add(N)` → `check_bounds` → +`.get(..)` → `try_into`. The widest attacker-influenced values in the +crate are `read_bytes_indirect`'s absolute `{offset,length}` pair +(`src/data_access.rs:328-350`). + +- **Drives:** the `pub` read functions directly with the fuzzer + choosing buffer, offset (including far-past-end and huge values), + and endianness; lanes for `read_bytes`/`read_string` (u32 length + prefix), `read_bytes_indirect`/`read_string_indirect` (the + `{offset,length}` pair), `read_enum`, `read_bool` strictness, and + each fixed-width kind from the macro family. +- **Invariants:** + - no-panic for any (buffer, offset, endian) triple; + - `bool` accepts exactly 0x00/0x01 and rejects everything else + (`:134-144` — the strictness is contract, pin it); + - invalid UTF-8 in `read_string` errors (`Access`), never a lossy + silently-corrupting parse (`:195-208`); + - bounds partition: an error implies `checked_add`-overflow or + `end > buffer_len` with the offending `field_path` named; an Ok + implies the field sits fully inside the buffer; + - nothing before/end-of-buffer is read: the decode consumes + exactly its declared width (offset unchanged on error paths); + - `read_bytes_indirect`'s data region always satisfies + `data_offset + data_length ≤ buffer_len` on Ok, and neither + field can push arithmetic past the buffer without an error + (the two `u32` widening casts at `:334, :341` widening-only, + verified by the partition). + +### Target 3 — `read_opseq` (stateful, wave 2) + +`SequentialReader` is stateful against attacker bytes (mutable cursor: +`field_index`, `position`; `src/sequential_reader.rs:144-148`) and +fuzzer-reachable operations are `read_next`, `read_next_borrowed`, +`read_field` (out-of-order names), `reset` (`:157-300`). + +- **Drives:** `#[derive(Arbitrary)]` op sequences (Next, Field(name + choice), Reset, End) against a compiled plan — the plan built once + per exec from a fixed small schema menu, bytes adversarial. +- **Invariants:** + - no-panic over any op interleaving and any buffer; + - cursor discipline: a failed read leaves the reader usable (a + subsequent `reset` restores the exact initial state; cursor never + exceeds the buffer); + - `read_next` returns fields exactly in plan order and `None` + exactly at plan end; interleaved `read_field` for any field at + any cursor state never panics and never mutates the sequential + cursor (its offset argument comes from the plan, not the reader); + - record-count spin bound: wire-controlled `count` loops + (`src/materialize.rs:439-442`, `:817-820`, + `src/sequential_reader.rs:985-988`) consume ≥ 4 verified bytes per + iteration, so iterations are bounded by + `remaining_bytes / 4` — a hostile count fails fast with `Access` + (encode as an explicit per-exec assertion, not just + no-panic/OOM); + - engine-issued readers are independent: two readers over the same + plan and buffer never observe each other's cursors + (ADR-007's owned-fresh-reader contract). + +### Target 4 — `layout_build` (wave 2) + +`LayoutBuilder::new` parses once (`src/layout_builder.rs:154-156`); +`build(&HashMap)` (`:189`) is repeatable with +attacker-shaped `var_sizes` driving write-position arithmetic in +`walk_struct`. + +- **Drives:** a fixed schema menu containing every variable-width + encoding × `#[derive(Arbitrary)]` `var_sizes` maps and write values + (`FieldValue` shapes). +- **Invariants:** + - no-panic across adversarial size maps (zero, huge, mismatched + with `max_length`/`count` declarations); + - every failed write leaves the buffer untouched (byte-equal to the + pre-call snapshot) or documented-partial exactly where the + contract allows — pin the actual contract the code implements; + - field positions from a successful `build` are disjoint and + in-bounds for the reported total size; + - `data_offset/length` pairs written by + `write_string_indirect`/`write_bytes_indirect` always satisfy the + read-side `read_*_indirect` bounds partition above — the write + side and the read side of the pair are one contract + (round-trip pair; `:373-417` guards verified by + `:733-750`-style assertions). + +### Target 5 — `validate_pair` (two-input structured, wave 3) + +The integration target: schema and bytes are both adversarial. + +- **Drives:** `#[derive(Arbitrary)]` (doc, bytes) — compile once per + exec with whichever mode the fuzzer picks, then drive + `validate_bytes`, `read_field` (arbitrary field paths, including + junk paths), `materialize_packed`/`materialize_aligned`, and + `read_next` under the compiled plan. +- **Invariants:** + - no-panic for any (doc, bytes) pair, either mode; + - validate/read/materialize agreement lattice: `validate_bytes` Ok + ⇒ `materialize_*` Ok and every `read_field` over a declared path + Ok; `materialize_*` error ⇒ `validate_bytes` error on the same + buffer (exact agreement direction pinned per the code's actual + contract — determine the strict/loose ordering from the + `validate_bytes` implementation, don't assume); + - non-finite floats (NaN/Inf) surfaced by `materialize` are always + `Access` errors, never silently `Null`/`0.0` + (`src/materialize.rs:876-883`); + - unknown field-path strings always error with `Access` naming the + path, never panic, never index the map by substring drift; + - `Value` output is serde-safe: `materialize_*` output round-trips + through `serde_json::to_vec` and back to a structurally equal + `Value` (structural only — this crate's serde_json builds with + `preserve_order`, so object key order is preserved; byte-identity + round-trips are acceptable only where the docs say lossless, + per the alkcall §7.3 false-positive trap when they don't). + +## 4. Corpus policy + +Committed hand-made seeds per target, generated by +`fuzz/gen_fuzz_seeds.py` (deterministic, in-tree, quiche pattern); +grown corpora and artifacts gitignored. Seed menus: + +- `bast_compile`: a minimal valid packed doc and aligned doc; every + `AlkTypeKind` once; each reject class (`align` 4097/65536/u32-max, + `count` over cap, `maxLength` over cap, depth-129 nesting both + inline-nested and via `$ref` chains, `$ref` cycle, `$ref` to + missing def, root not a struct, missing `type`, unknown kind + string, duplicate field names first-wins probe, non-object doc, + deeply-nested JSON at serde_json's own 128 limit); `json.dict` + carries the BAST token set. +- `data_access`: minimal valid encodings per kind per endianness; + truncation at every prefix length (1..N-1 for each width); + `len = 0` / `MAX_LENGTH` / `u32::MAX` prefixes; the indirect pair at + {0,0}, {len, big}, {big, 0}, {u32::MAX, u32::MAX}; offset one-past- + end, offset u32-magnitude; 0x02 bool byte; invalid UTF-8 in a + string; enum value out of range; NaN/Inf bytes. +- `read_opseq` / `layout_build` (wave 2): the semantic fixtures — + full sequential walk, reset-mid-walk then full walk again, failed + read then reset, record loop with a hostile count under a real + buffer, junk field paths; zero/huge/mismatched `var_sizes`; + overwrite-everything write; indirect-pair overflow write, plus the + write-then-read pair fixture. +- `validate_pair` (wave 3): hostile-schema/valid-bytes, + valid-schema/hostile-bytes, valid/valid — plus the §6 candidate + shapes as pinned reproducers. + +Stateful `Arbitrary` seeds are hand-encoded against the `arbitrary` +1.4.x derive layout with per-element keep-going bytes, pinned by +seed-decode tests (alkhttp's target-4 pattern). + +## 5. Campaign + gate policy + +Identical to the siblings (alkcall §7.9 posture; no hosted CI in this +repo): + +- **Corpus replay is the standing fuzz gate:** `cargo test + --manifest-path fuzz/shared/Cargo.toml` — joins AGENTS.md's + verification checklist. +- **Campaigns run detached** via `fuzz/run-detached.sh`; budget 10 min + per target for a smoke campaign, 30–45 min before a release or after + touching `src/data_access.rs`, `src/schema.rs`, the compile walks, or + the sequential reader. +- **Grown corpora stay gitignored** (hash-named files ignored via + pattern; committed `seed-*` files stay); merge worthy entries into + seeds only deliberately. +- libFuzzer flags worth pinning: `-rss_limit_mb=2048`, + `-malloc_limit_mb=2048`, `-timeout=25`, `-max_len=65536`, + `-use_value_profile=1`, and `-dict=json.dict` on the JSON targets + (alkcall §7.2's set, minus the CI-tier concerns). +- Nightly stays confined to `fuzz/`; the main crate's stable build, + MSRV, and wasm target are untouched — `cargo fuzz build` must never + be a prerequisite for `cargo test`/`clippy`/`build`. + +## 6. Pre-fuzzing candidate findings (confirm or refute) + +These are pre-fuzzing code-review findings from the 0.3.0 inventory, +verified against the code. They define what the targets must encode as +invariants and are the first corpus entries to add; the fuzzer +confirms or refutes them. + +1. **Wire-controlled `Record` count loops** (`src/materialize.rs:439- + 442`, `:817-820`, `src/sequential_reader.rs:985-988`): the only + buffer-derived loop counts (`read_u32(..)? as usize` then + `for i in 0..count`). Each iteration performs at least one + bounds-checked read, so a hostile count should fail fast with + `Access` — bounded by `remaining_bytes / 4`, no allocation, no + spin. Correct as designed *if and only if* that holds; the + `read_opseq` target encodes it as an explicit assertion (§3 + target 3) so the fuzzer can break it the moment any per-entry + cost stops being `≥ 4 verified bytes`. +2. **Attacker-controlled absolute `{offset,length}` pairs** + (`read_bytes_indirect`/`read_string_indirect`, + `src/data_access.rs:307-350`): the widest attacker-influenced + values in the crate (each `u32`, up to 2³²−1). The pattern + (widening cast → `checked_add` pair-sum → full bounds check) + looks correct; campaign confirms the bounds partition on every + (buffer, pair) input, both sides of the write/read contract + (§3 target 2 / target 4). +3. **Duplicate field-path tolerance by design** (`OffsetMap::build_ + index`, `src/offset_map.rs:199-205`: first-wins; `BastStruct:: + parse` does not reject duplicates): ambiguous lookups are + documented behavior. Encode as an invariant — a successful + engine's `read_field` resolves duplicates deterministically + (first wins) — so a future "reject duplicates" change shows up + as a deliberate contract change, not silent drift. +4. **`align_up`/`round_up` plain `+` arithmetic** (`src/offset_map.rs: + 618-639`): un-checked `+ align - rem` in a field of + schema-controlled values. Overflow-infeasible today because + align is capped at parse (`MAX_ALIGN` = 4096) and the running + offset is monotonically checked — a `bast_compile` corpus entry + with align at the cap pinning the boundary keeps it that way if + the cap ever moves. +5. **`bast_validation::validate_value` recompiles a `ValidationPlan` + per call** (`src/bast_validation.rs:64-65`): a repeated-op DoS + surface if any consumer compiles-per-call. Not a bug in this + crate's API; fuzz targets compile once per exec, and the doc + records the compile cost as the consumer's responsibility. + +None of these rises to the alkcall §6.2 / alkhttp FWD-20 class; they +are boundary-confirmations, which is exactly the expected profile of +this crate (§1 honest caveat). + +## 7. Sequencing + +1. **Wave 1** — `cargo fuzz init`, infra (`workspace` exclude, + toolchain pin, runner, seed generator, README, `.gitignore`), + targets 1–2 + corpora + corpus replay + AGENTS.md gate + smoke + campaigns (10 min per target, detached). +2. **Wave 2** — targets 3–4 (stateful `read_opseq`, `layout_build`) + + seeds + smoke campaigns. Add regression fixtures for anything + wave 1 surfaced first. +3. **Wave 3** — target 5 `validate_pair` (the two-input structured + harness) + long (45 min) release-budget campaigns across all + targets; merge any curated inputs into seeds deliberately. +4. Everything else inherited verbatim: no hosted CI, OSS-Fuzz out, + no Actions/workflow files anywhere in the repo (alkcall §7.9). + +## 8. References + +- alkcall `docs/research/fuzzing.md` — rationale, tool landscape, + campaign containment (§7.6), no-hosted-CI policy (§7.9) +- alkhttp `docs/plans/fuzzing.md` — the live pattern this plan copies + (waves, findings log, `fuzzing` hub convention) +- RUSTSEC-2026-0037 / CVE-2026-31812 (quinn-proto) — the remote-DoS + class this crate's peers are exposed to +- Internal: ADR-002 (layout modes), ADR-004 (error/validation + strategy), ADR-006 (aligned-mode variable-field rejection), ADR-008 + (aligned-mode TUnion rejection), ADR-010 (`validate_bytes`, + materialize-then-validate) \ No newline at end of file