The running validate_pair campaign found a third crash: in aligned
mode a maxLength reservation (ADR-003 strategy 2, VARCHAR(N)) stores
RAW zero-padded data with no length prefix — materialize and
validate_bytes implement exactly that — but read_field read the entry
through data_access::read_string, i.e. parsed the window's first four
bytes as a u32 length prefix. Raw reservation bytes that look like a
large prefix then fail bounds with Access while validate_bytes says
Ok: the validate⇒read agreement lattice breaks on every aligned
maxLength string/bytes field (any schema declaring maxLength in
aligned mode). write_field had the same mismatch (prefix+data into a
raw window).
Engine fix:
- VariableEncoding gains MaxLengthReserved (additive variant, ADR-003
strategy 2). OffsetMap::compute records it for maxLength fields with
the default encoding; maxLength+offset-indirect stays OffsetIndirect
(the pair read is intentional, the window reserves max_len bytes),
preserving the W3-1 combination semantics.
- read_field String/Bytes arms dispatch on MaxLengthReserved → new
data_access::read_reservation_string / read_reservation (raw window
inside-buffer check + NUL trim — the materializer's exact semantics).
- write_field dispatches → new data_access::write_reservation (zero-
pads the window, rejects oversized values with Access).
- materialize_aligned reads MaxLengthReserved through the same new
read_reservation paths (single source of truth; replaces the inline
trim logic with an identical implementation).
- offset_map compute rejects a MaxLengthReserved encoding reaching the
walk with a clean Offset error (recorded, never declared).
- builder round-trips: MaxLengthReserved serializes via maxLength (the
document form), never as an encoding value.
- three engine regression tests: raw-not-prefixed read, zero-pad
write + oversize rejection, validate⇒read_field agreement.
- fuzz/shared validate_pair invariant updated: the W3-1
shorter-than-reservation exemption now applies to offset-indirect
only; reservations assert the full window in-bounds (fixed engine).
- corpus regenerated for generator-consistent numbering (seeds 037-044
relabeled; W3-1/W3-2 artifacts remain 044/045-047 → now 044, 048-050
region) — 48 seeds, replay 30/30 green.
Verification: main crate 573 tests pass; clippy -D warnings clean
(crate + shared); wasm clean; cargo fuzz build clean.
The restarted validate_pair campaign found a second crash: materialize
produces f64 0x5bffffffffffffff, serde_json emits the shortest repr
1.4536774485912136e+135, and the non-`float_roundtrip` parse side
(lexical concise-float over re-parsed digits) lands one ulp low —
probe-verified upstream of this crate (ryu's own float parser accepts
the same digits exactly; the std parser is exact; only serde_json's
concise reparse drifts). The harness's structural serde round-trip
assertion assumed Value equality holds for every finite f64 — upstream
parse-side drift breaks that assumption on adversarial magnitudes.
- assert_values_agree_with_ulp_slack replaces the bare Value equality:
keys/shapes exact, numbers equal-or-within-one-ulp (bit diff ≤ 1)
- the exact artifact bytes pinned as corpus seed-047, plus
deterministic minimal forms as seed-045/seed-046 (45→48 seeds)
- upstream note: enabling serde_json's float_roundtrip feature would
remove the drift; the crate pins serde_json default features +
preserve_order by design, so the slack is the honest pin
Verification: corpus replay 30/30 green (48 seeds), fuzz build clean,
clippy -D warnings clean.
The running validate_pair campaign found the first wave-3 crash
(artifact crash-e40d...): the harness invariant 'validate_bytes Ok ⇒
every offset-map leaf's range.end ≤ buffer.len()' is WRONG for
offset-indirect entries. In aligned mode a maxLength reservation
contributes its full window to the layout (menu 2's total is 68), while
the {data_offset, data_length} pair is absolute — the data may live
anywhere in the buffer and the all-zero pair {0,0} over a 64-byte
buffer validates and reads fine. The wave-1 data_access bounds
partition is the real contract; the new invariant exempted the read
side but over-asserted the window. Harness-bug, not engine-bug.
- invariant now splits: non-indirect leaves keep the full window
assertion; offset-indirect leaves assert only the read_ok ⇒ pair
agreement (the pointed-to window sits inside the buffer)
- the exact artifact bytes pinned as a regression test
(validated_buffer_may_be_shorter_than_the_indirect_reservation_window)
and as corpus seed-044; the generator emits the same shape
deterministically (44→45 seeds)
- PairInput fields made pub for out-of-crate triage probes
Verification: corpus replay 30/30 green; clippy -D warnings clean.
Target 5 (§3): compile an attacker schema (10-lane menu incl. raw JSON
bytes lane) in both modes, then hammer the hostile buffer through
validate_bytes, an independent materialize_packed/materialize_aligned,
read_field over every offset-map leaf, junk field paths, and the packed
sequential walk under the spin bound.
Invariants coded (per §3 target 5): mode agreement (aligned Ok ⇒ packed
Ok; packed-Ok/aligned-Err only for the documented ADR-006/ADR-008/
offset-indirect rejections), the materialize⇄validate_bytes verdict
lattice with verbatim error propagation, unknown-path echo, serde
round-trip of materialized output, non-finite-float Access pinning,
out-of-range enum Validation pinning, and the record-count spin bound.
44 committed seeds (menu/raw lanes × valid/valid, hostile-schema/
valid-bytes, valid-schema/hostile-bytes incl. a per-prefix truncation
sweep, NaN/Inf, enum 99, spin fixtures, mode-agreement pins), hand-
encoded against the pinned arbitrary 1.4.2 derive layout and pinned by
decode tests. The aligned maxLength-reservation offset pin (s@8..72,
tail@72, total 76) caught a fixture assumption error pre-commit.
Verification: corpus replay 29/29 green (44 new seeds decode+replay),
main crate 570 tests pass, clippy -D warnings clean (crate + shared),
wasm clean, cargo fuzz build clean. Hand-run drives (indirect pair
escape, enum-Validation, unknown discriminator, trailing garbage) all
held.
Targets 3-4 of docs/plans/fuzzing.md, per the sibling layout:
- fuzz/shared/src/read_opseq.rs — SequentialReader op sequences
(Next/NextBorrowed/Field/Reset/End, Arbitrary-derived) over hostile
buffers under the fixed packed schema menu. Invariants: cursor
discipline (failed read leaves position untouched, state replay
deterministic), None sticky at plan end, plan-order full walks with
a spin bound, read_field leaves a usable reader, ADR-007 reader
independence (shared Arc, isolated cursors), and the plan §6-1
record-count ≥4-verified-bytes bound encoded as an explicit End-op
assertion.
- fuzz/shared/src/layout_build.rs — LayoutBuilder::build with
adversarial var_sizes over a five-schema menu (string/bytes, nested
struct, byte-disc union, record+array, fixed control). Invariants:
Offset-class failures only, position disjointness + total-size
bounds, variable fields record their 4-byte prefix, failed writes
leave the buffer byte-identical, write→read pair round trip.
- derive_var_sizes discovers the synthetic keys ('p.__discriminator')
the builder actually wants by parsing the quoted key from the
Offset reason.
- 73 committed seeds (58 read_opseq + 15 layout_build) hand-encoded
against the pinned arbitrary 1.4.2 derive layout (4-byte LE
multiply-shift variant selectors, keep-going vec elements,
take-rest last field) and pinned by decode_lands_on_the_intended_variants
replay tests; gen_fuzz_seeds.py mirrors the encoders.
- Engine fix (finding W2-1): plan_read_array returned Ok for a
fixed-stride array whose count*stride window extended past the
buffer — the struct/union arms bounds-check, the array arm did not;
a truncated array deferred the failure to the next field (wrong
path) or masked it entirely as an Ok walk. Now an Access error
naming the array, regression test in sequential_reader.rs.
- Packed-mode 'encoding: offset-indirect' pinned as the documented
inline-length-prefix no-op (finding W2-2, bast-format.md Default
strategy selection); open design question recorded as plan §6-7.
Verification: fuzz corpus replay 19/19; main crate 570 tests incl.
the new regression; clippy -D warnings clean (crate + shared); wasm
build clean; cargo fuzz build clean (nightly confined to fuzz/).
Smoke campaigns (10 min detached each): read_opseq 52.1k execs exit 0
empty artifacts, layout_build 42.4k execs exit 0 empty artifacts; no
crash/oom/timeout on any fork job.
glm-5.3-flash