diff --git a/README.md b/README.md index 3b38341..506bb86 100644 --- a/README.md +++ b/README.md @@ -269,7 +269,8 @@ Architecture documentation lives under [`docs/architecture/`](docs/architecture/ (ADR-003), error handling (ADR-004), int64/uint64 kinds (ADR-005), non-final inline variable fields (ADR-006), packed-mode read factory (ADR-007), TUnion in aligned mode (ADR-008), builder API (ADR-009), - `validate_bytes` (ADR-010) + `validate_bytes` (ADR-010), compiled read plan (ADR-011), plan + fingerprinting + `ValidationPlan` (ADR-012) ## License diff --git a/docs/architecture/README.md b/docs/architecture/README.md index d5bbe7f..45d739b 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -46,7 +46,7 @@ format definition; the engine is generic. | [009](decisions/009-builder-api.md) | Builder API for Schema Construction | Fluent Rust API producing `serde_json::Value`; covers BAST kinds + standard JSON Schema; resolves OQ-003. *Output format amended to BAST / standard JSON Schema by ADR-BAST.* | | [010](decisions/010-generalized-validation-validate-bytes.md) | Generalized Validation — `validate_bytes` on `AlkTypeEngine` | Single-call binary-buffer validation; materialize `Value` from bytes, then validate. *Validation step amended to the BAST-native validator by ADR-VAL-SPLIT.* | | [011](decisions/011-compiled-read-plan-for-packed-mode.md) | Compiled Read Plan for Packed Mode | `ReadPlan` — the packed read-side compiled form, symmetric to `OffsetMap` (aligned) and `PackedLayout` (packed write). Closes review #004's 400x read-path gap; retires ADR-007's "re-parse on demand" framing. *Accepted.* | -| [012](decisions/012-plan-fingerprinting-and-m1-closure.md) | Plan Fingerprinting, ValidationPlan, and Closing the Deferred M1 Sites in 0.3.0 | `ReadPlan`/`OffsetMap`/`ValidationPlan` `Hash + Eq` + `fingerprint()`; owned `BastDoc` (lifetime removal); `OffsetMap` carries `LeafMeta` to close the aligned-side M1 sites; `ValidationPlan` retires the interpretive `bast_validation` walk (review #005 M3 reversed the original deferral). Bundles with ADR-011 into one 0.3.0 breaking release. *Proposed.* | +| [012](decisions/012-plan-fingerprinting-and-m1-closure.md) | Plan Fingerprinting, ValidationPlan, and Closing the Deferred M1 Sites in 0.3.0 | `ReadPlan`/`OffsetMap`/`ValidationPlan` `Hash + Eq` + `fingerprint()`; owned `BastDoc` (lifetime removal); `OffsetMap` carries `LeafMeta` to close the aligned-side M1 sites; `ValidationPlan` retires the interpretive `bast_validation` walk (review #005 M3 reversed the original deferral). Bundles with ADR-011 into one 0.3.0 breaking release. *Accepted — §3a shape implemented in `src/validation_plan.rs`.* | ## Relevant Open Questions diff --git a/docs/architecture/bast-format.md b/docs/architecture/bast-format.md index 4ce1c6b..df7e5a1 100644 --- a/docs/architecture/bast-format.md +++ b/docs/architecture/bast-format.md @@ -528,24 +528,28 @@ materializer iterates the field list), types are correct (`read_u32` produces `Value::Number`), bounds are checked (via `data_access:: check_bounds`), UTF-8 is valid (via `from_utf8`), the discriminator is in the mapping, and the boolean byte is 0 or 1. What the materializer -does NOT check — and what the 19 v0.1.0 custom keyword validators check -afterward — are **value-domain constraints expressed in the BAST -document**. The BAST-native validator is a recursive walker over the -BAST type tree that checks exactly these: +does NOT check — and what the validation half checks afterward — are +**value-domain constraints expressed in the BAST document**. Under +ADR-012 §3 these constraints are compiled once into a `ValidationPlan` +at engine-compile time (eager `$ref` resolution, cyclic-graph +rejection); each `validate_bytes` call walks the compiled constraint +tree against the `Value`. The plan's nodes enforce exactly these +constraints (the set is normative; the walker that enforced it +interpretively in 0.2.0 is retired): -| Constraint | Validator arm | -|------------|---------------| -| Integer range (Int8..Uint64) | `validate_int`/`validate_uint` with `as_i64`/`as_u64` + range check | -| Int64/Uint64 (full range) | `validate_int64`/`validate_uint64` (JSON precision caveat per ADR-005) | -| Float finiteness (Float32/64) | `validate_float` with `as_f64().is_finite()` | -| String `maxLength` (byte length) | `check_string` reads the field-level `maxLength` | -| Bytes `maxLength` (array length) | `check_bytes` accepts the `Value::Array` form (the materializer emits bytes as an array of u8) | -| Enum index bounds | `validate_enum` checks `idx < values.len()` — **fixes the v0.1.0 dead constraint** | -| Union variant dispatch | `validate_union` reads `__discriminator`, looks up the variant, recurses via `validate_typeref` | -| Struct fields | `validate_struct` walks `fields`, requires each declared field present, recurses | -| Array count | `validate_array` checks `arr.len() == count` and recurses per element | -| Record values | `validate_record` recurses into each value's `values` type | -| Boolean | `validate_bool` (materializer already rejects non-0/1 bytes) | +| Constraint | `ValidNode` arm | +|------------|-----------------| +| Integer range (Int8..Uint32) | `Int { min, max }` / `Uint { max }` with `as_i64`/`as_u64` + range check | +| Int64/Uint64 (full range) | `I64` / `U64` (JSON precision caveat per ADR-005) | +| Float finiteness (Float32/64) | `Float` with `as_f64().is_finite()` | +| String `maxLength` (byte length) | `Str { max_len }` — `maxLength` baked in from the owning field at compile time | +| Bytes `maxLength` (array length) | `Bytes { max_len }` — accepts the `Value::Array` form (the materializer emits bytes as an array of u8) | +| Enum index bounds | `Enum { count }` checks `idx < count` — **fixes the v0.1.0 dead constraint** | +| Union variant dispatch | `Union { variants }` reads `__discriminator`, dispatches on the compiled variant nodes | +| Struct fields | `Struct { fields }` requires each declared field present, recurses | +| Array count | `Array { count, element }` checks `arr.len() == count` and recurses per element | +| Record values | `Record { values }` recurses into each value | +| Boolean | `Bool` (materializer already rejects non-0/1 bytes) | No external JSON Schema is required for `validate_bytes`. The BAST document is the complete specification of the binary format — it @@ -584,7 +588,8 @@ The `jsonschema` crate **remains a direct dependency** for meta-schema. The only thing removed is the custom keyword integration path. The `validate_bytes` path no longer touches `jsonschema` — a small wasm binary-size win in addition to the architecture -simplification. +simplification. (Since ADR-012 §3, the interpretation step itself is +also compiled away: see the `ValidationPlan` above.) ### `AlkTypeError::Validation` payload shape diff --git a/docs/architecture/decisions/012-plan-fingerprinting-and-m1-closure.md b/docs/architecture/decisions/012-plan-fingerprinting-and-m1-closure.md index 2031a06..2bee8a5 100644 --- a/docs/architecture/decisions/012-plan-fingerprinting-and-m1-closure.md +++ b/docs/architecture/decisions/012-plan-fingerprinting-and-m1-closure.md @@ -2,7 +2,8 @@ ## Status -Proposed. Bundles three pieces of work into the 0.3.0 release so the +Accepted. Implemented (§3's `ValidationPlan`, 0.3.0 phase 7, 2026-08-31); +bundles three pieces of work into the 0.3.0 release so the crate ships one round of breaking changes, not two (or three). The three pieces: (a) fingerprinting `ReadPlan`/`OffsetMap`, (b) closing the deferred M1 sites via an owned `BastDoc` + `OffsetMap` `LeafMeta`, @@ -11,6 +12,8 @@ and (c) a `ValidationPlan` that retires the interpretive `ValidationPlan`" decision — see "ValidationPlan — in scope for 0.3.0" below). Companion to [ADR-011](011-compiled-read-plan-for-packed-mode.md) (the `ReadPlan`) and the [0.3.0 implementation plan](../../plans/030-compiled-forms.md). +§3's concrete shape was scoped by the follow-on design session and is +implemented in `src/validation_plan.rs` — see §3a below. ## Context @@ -340,6 +343,69 @@ plan or stays a thin match over plan-carried descriptors. These are shape questions, not decision questions; the decision (in 0.3.0, compiled form, no per-buffer `BastDoc` walk) is fixed here. +### 3a. `ValidationPlan` shape — resolved by the design session + +The follow-on design session (0.3.0 phase 7 predecessor) resolved the +open shape questions; implemented in `src/validation_plan.rs`: + +- **Shape.** `ValidationPlan { root: ValidNode }`, a compiled + constraint tree — one `ValidNode` arm per value-domain check, + mirroring the interpretive walker's arms one-to-one: + `Int { min, max }` / `I64` / `Uint { max }` / `U64` / `Float` / `Bool` + / `Str { max_len }` / `Bytes { max_len }` / `Enum { count }` / + `Struct { fields: Vec }` / `Union { variants: + Vec }` / `Array { count, element }` / `Record + { values }`. `ValidField` carries `name + node`; `ValidVariant` + carries `key + node`. The nodes are public (diagnostics access via + `ValidationPlan::root()`); construction is only possible through + `compile`. Note the union node carries *only the variant nodes* — the + declared union `fields` (shared fields) are validated as part of the + variant walk, because the walker dispatches on the materialized + `__discriminator` and validates the whole object against the selected + variant (the `ValidNode::Union` doc comment records this; the + interpretive union arm recursed into the variant the same way). +- **Constraint representation.** Inline scalar fields on the node arms + (ranges as `i64`/`u64` pairs, `maxLength` as `Option`, enum + bound as `count: u64`, union keys as owned `String`s). `maxLength` is + resolved from the *owning field* at compile time and baked into the + `Str`/`Bytes` leaf — the walk never consults field annotations. It + never crosses a `$ref` (a `$ref` always targets a struct/union/enum + `$defs` entry, so the interpretive walk could never consult it + through one either). +- **`compile` signature.** `ValidationPlan::compile(&BastDoc) -> + Result` — the plan-table's `&str` root-name + parameter was vestigial (the doc already holds its root). +- **`validate_value` disposition.** Retained as a one-shot wrapper: + `compile(doc)` + `validate(value)`. The interpretive walker behind it + is *retired* (deleted) — the wrapper delegates to the plan, so there + is one constraint implementation, not two. `bast_validation.rs` keeps + the shared error helper (`validation_err`) and the + `__discriminator` key constant. +- **Error contract preserved.** The plan walk reproduces the + interpretive error messages byte-identically: a segment stack + (`field` / `[index]` / `[key]`) renders paths only on failure — zero + per-node allocation on the happy path. Numeric `__discriminator` + dispatch matches mapping keys without allocation for the u64/i64 + forms (mapping keys are stringified integers; non-integer numbers + fall back to `Number::to_string`). +- **Compile-time rejection of adversarial graphs.** Eager `$ref` + resolution with a definition-level cycle set and a depth cap (128): + a cyclic or self-referential schema is `AlkTypeError::Schema` at + compile, not a stack overflow — the interpretive walker resolved + `$ref`s lazily with no guard and could overflow on recursion. + Diamond (shared, non-cyclic) refs compile fine; the cycle set is + path-scoped. +- **Engine integration.** `AlkTypeEngine` holds `Arc` + built at `compile` time in *both* modes; the accessor + `validation_plan() -> &Arc` is new public API. + `validate_bytes`'s signature is unchanged. The plan compile runs + *before* the layout build: it is the engine's reference-graph gate + (see Consequences). +- **Scope note.** phase-7's `Send + Sync` / `Hash + Eq` / + `fingerprint()` requirements are structural on the types above + (`#[derive(...)]` on plain owned data; the same `DefaultHasher` + fingerprint as phase 6). + ### 4. Fingerprinting `OffsetMap` (bundled with §2b) Since `OffsetMap` is getting new fields (`LeafMeta`) in §2b, its @@ -380,11 +446,16 @@ aligned reads/writes over identical bytes. *contract* and method are in scope (§1, §3); the downstream uses (cache format, wire protocol) are the consumers' problem, not this ADR's. -- The concrete `ValidationPlan` struct/enum shape and constraint - representation — scoped in the 0.3.0 implementation plan's - dedicated phase and a follow-on design session (see §3 "What this - ADR does *not* decide"). The decision (in 0.3.0, compiled form, no - per-buffer `BastDoc` walk) is fixed; the shape is not. +- The `ValidationPlan` shape — **resolved** (§3a). Decided by the + design session and implemented in `src/validation_plan.rs`; the + decision (in 0.3.0, compiled form, no per-buffer `BastDoc` walk) was + fixed here. +- Cycle-guard hardening for the *layout* walkers' own recursion + (`LayoutBuilder`/`OffsetMap` struct recursion) beyond the engine-path + gate described in Consequences — if a non-engine entry point walking + those types on untrusted docs becomes a consumer pattern, the + guards get their own change (the `AlkTypeEngine::compile` gate + covers the supported path today). - Cross-version fingerprint stability — the fingerprint is stable within a crate version but may change across versions (a new `AlkTypeKind` variant, for example, changes the hash). Cross-version @@ -412,7 +483,23 @@ aligned reads/writes over identical bytes. half was not. Closing it here — while there are zero real consumers and one breaking bump already paying the downstream-churn cost — avoids a second breaking change to `validate_bytes`/`bast_validation` - after 0.3.0. + after 0.3.0. (Implemented: a spot benchmark of plan-validate on a + 4-field mixed frame puts the validation half at ~0.2 µs/validate; + the compile-per-call one-shot it replaces runs ~2.7x slower before + the walk is even counted — and the full 0.2.0 per-buffer cost + included lazy `$ref` deep-clones that the one-shot no longer pays. + The materialize half, not validation, remains the dominant + `validate_bytes` cost.) +- **Compile-time rejection of cyclic `$ref` graphs.** A side effect of + eager plan compilation: a self-referential document is now a clean + `Schema` error instead of a stack overflow. The plan compile runs + *before* the layout build in `AlkTypeEngine::compile`, making it the + engine's reference-graph gate — `LayoutBuilder`/`OffsetMap` + struct-recursion has no cycle guard and previously could recurse + unboundedly on such a document (a pre-existing untrusted-schema + hazard, surfaced by the phase-7 `compile_rejects_cyclic_ref_graph` + test). Hardening the layout walkers' own recursion is a separate + cleanup, not required while the gate holds in the engine path. - **Fingerprinting enables downstream uses.** Cross-run plan caching, `alkcall` schema handshake, and schema-version diagnostics all become possible without further API work — across `ReadPlan`, @@ -431,8 +518,9 @@ aligned reads/writes over identical bytes. - **Breaking public-API changes (0.2.0 → 0.3.0).** `BastDoc<'a>` → `BastDoc` (owned) changes every `Bast*` signature that took `&'a`. `OffsetMap::get` return type changes. `LeafMeta` is new public. - `ReadPlan` is new public (from ADR-011). `ValidationPlan` is new - public (§3). All ride the bump. + `ReadPlan` is new public (from ADR-011). `ValidationPlan` (+ the + `ValidNode`/`ValidField`/`ValidVariant` node types) is new public + (§3a). All ride the bump. - **`BastDoc` ownership refactor is broad.** Touches `bast.rs` (every typed node: `&'a str` → `String`/`Arc`, `&'a Value` → `Value`/`Arc`) and every consumer (`layout_builder`, @@ -442,13 +530,21 @@ aligned reads/writes over identical bytes. anyway. The refactor is mechanical (lifetime removal, not logic rewrites); the POC on `readplan-poc` confirmed the read path is unaffected. -- **`ValidationPlan` shape work is not yet scoped.** §3 fixes the - decision (in 0.3.0, compiled form) but defers the concrete shape to - a follow-on design session + a dedicated plan phase. This is a - tracked, owned deferral with a concrete reactivation trigger (the - shape session before phase 7), not a black-hole hedge: the - implementation phase is committed in the plan, so the work cannot - slip past 0.3.0 without reopening this ADR. +- **Interpretive `validate_value` is compile-per-call.** The retained + one-shot wrapper (`bast_validation::validate_value`) compiles a plan + then validates — fine for one-off/diagnostic use, wrong for per- + buffer use. Per-buffer callers must hold the engine (or a plan) — + the doc comments say so. The walker it replaced had the inverse + trade (no compile, but interpretive per call); the engine path + (compile once) is the one that matters. +- **Aligned-mode `maxLength`-reserved strings/bytes.** Materialization + emits the *full reserved* (zero-padded) data for these fields. A + `ValidationPlan` compiled from a document used in packed mode would + apply `maxLength` to trimmed length, matching packed semantics; the + aligned materializer's zero-padding means the value passed to + validation can carry trailing NULs. This is pre-existing + materialize behavior (not a plan artifact); consumers relying on + trimmed values already see it. - **Fingerprint cross-version stability is not guaranteed.** A future `AlkTypeKind` variant changes the hash. Documented as a within- version contract. Consumers that need cross-version stability @@ -463,8 +559,7 @@ aligned reads/writes over identical bytes. M1 fixes are "cache the parse" (§2a) and "extend the compiled form with leaf metadata" (§2b), not "add a third compiled form." - **Not a `ValidationPlan` deferral.** `ValidationPlan` is in scope - (§3). The shape is to be scoped in a follow-on session; the - decision to ship in 0.3.0 is fixed. + (§3) and implemented (§3a); the interpretive walker is retired. - **Not cross-version fingerprint stability.** Within-version only. - **Not a disk-cache or wire-protocol spec.** The fingerprint contract and method are in scope; the downstream uses are the consumers' diff --git a/docs/architecture/validation.md b/docs/architecture/validation.md index 463b072..9ee5771 100644 --- a/docs/architecture/validation.md +++ b/docs/architecture/validation.md @@ -1,6 +1,6 @@ --- status: accepted -last_updated: 2026-08-15 +last_updated: 2026-08-31 --- # alktype — Validation @@ -22,7 +22,7 @@ and recorded in [ADR-VAL-SPLIT](decisions/val-split-two-validator-model.md). | Path | Input | Validator | Schema source | |------|-------|-----------|---------------| -| `validate_bytes(&[u8])` | Raw bytes | BAST-native validator (`bast_validation`) | The BAST document (binary layout + value constraints) | +| `validate_bytes(&[u8])` | Raw bytes | Compiled `ValidationPlan` walk | The BAST document (binary layout + value constraints) | | `validate_json(&Value)` | Parsed JSON `Value` | Standard `jsonschema::Validator` | A consumer-provided standard JSON Schema | ### `validate_bytes` — bytes in, BAST is the validator @@ -35,25 +35,39 @@ produces `Value::Number`), bounds are checked (via `data_access::check_bounds`), UTF-8 is valid (via `from_utf8`), the discriminator is in the mapping, and the boolean byte is 0 or 1. -What the materializer does NOT check — and what the BAST-native -validator checks afterward — are **value-domain constraints expressed -in the BAST document**. The BAST-native validator -(`src/bast_validation.rs`) is a recursive walker over the BAST typed -tree ([`crate::bast::BastDoc`]/[`BastType`]) that checks exactly these: +What the materializer does NOT check — and what the validation half +checks afterward — are **value-domain constraints expressed in the BAST +document**. Since ADR-012 §3 (0.3.0), those constraints are not walked +interpretively per buffer: they are **compiled once** into a +`ValidationPlan` ([`src/validation_plan.rs`](../../src/validation_plan.rs)) +at `AlkTypeEngine::compile` time, and each `validate_bytes` call walks +the compiled constraint tree against the materialized `Value` — no +`$ref` re-resolution, no schema re-parse, no per-node path formatting +(error paths render only on failure). The plan's constraint nodes +implement exactly the table below (the constraint set is unchanged from +the retired interpretive walker): -| Constraint | Validator arm | -|------------|---------------| -| Integer range (Int8..Uint64) | `validate_int`/`validate_uint` with `as_i64`/`as_u64` + range check | -| Int64/Uint64 (full range) | `validate_int64`/`validate_uint64` (JSON precision caveat per ADR-005) | -| Float finiteness (Float32/64) | `validate_float` with `as_f64().is_finite()` | -| String `maxLength` (byte length) | `check_string` reads the field-level `maxLength` | -| Bytes `maxLength` (array length) | `check_bytes` accepts the `Value::Array` form (the materializer emits bytes as an array of u8) | -| Enum index bounds | `validate_enum` checks `idx < values.len()` — **fixes the v0.1.0 dead constraint** | -| Union variant dispatch | `validate_union` reads `__discriminator`, looks up the variant, recurses via `validate_typeref` | -| Struct fields | `validate_struct` walks `fields`, requires each declared field present, recurses | -| Array count | `validate_array` checks `arr.len() == count` and recurses per element | -| Record values | `validate_record` recurses into each value's `values` type | -| Boolean | `validate_bool` (materializer already rejects non-0/1 bytes) | +| Constraint | Plan node (`ValidNode`) | +|------------|-------------------------| +| Integer range (Int8..Uint32) | `Int { min, max }` / `Uint { max }` | +| Int64/Uint64 (full range) | `I64` / `U64` (JSON precision caveat per ADR-005) | +| Float finiteness (Float32/64) | `Float` with `as_f64().is_finite()` | +| String `maxLength` (byte length) | `Str { max_len }` — `maxLength` baked in from the owning field at compile time | +| Bytes `maxLength` (array length) | `Bytes { max_len }` — accepts the `Value::Array` form (the materializer emits bytes as an array of u8) | +| Enum index bounds | `Enum { count }` checks `idx < count` — **fixes the v0.1.0 dead constraint** | +| Union variant dispatch | `Union { variants }` reads `__discriminator`, dispatches on the compiled variant nodes | +| Struct fields | `Struct { fields }` requires each declared field present, recurses | +| Array count | `Array { count, element }` checks `arr.len() == count` and recurses per element | +| Record values | `Record { values }` recurses into each value | +| Boolean | `Bool` (materializer already rejects non-0/1 bytes) | + +The plan is a public type (`ValidationPlan`, `Debug + Clone + +PartialEq + Eq + Hash + Send + Sync`): `engine.validation_plan()` +exposes it for consumers that validate their own materialized `Value` +trees or want its `fingerprint()` for caching / schema handshakes +(ADR-012 §1). The one-shot `bast_validation::validate_value(&doc, +&value)` remains as a convenience wrapper (compile + validate) for +callers holding a BAST document without an engine. No external JSON Schema is required for `validate_bytes`. The BAST document is the complete specification of the binary format — it @@ -122,12 +136,14 @@ simplification. The strategy is decided in [ADR-004](decisions/004-error-handling-validation-strategy.md) and refined by [ADR-VAL-SPLIT](decisions/val-split-two-validator-model.md): -1. **Load time:** Parse the BAST document into the typed tree, compute +1. **Load time:** Parse the BAST document into the typed tree, compile + the `ValidationPlan` (the value-domain constraint tree), compute the layout engine, and (optionally) build the standard `jsonschema::Validator` for the JSON-validation path. This is the `AlkTypeEngine::compile` constructor. 2. **Access time:** Use the compiled engine for repeated read/write - operations. Validation is opt-in per operation. + operations. Validation is opt-in per operation: the validation half + walks the compiled `ValidationPlan`, never the BAST document. ### The `AlkTypeEngine` struct @@ -138,6 +154,7 @@ supports both layout modes (ADR-002) via an internal `Layout` enum: pub struct AlkTypeEngine { layout: Layout, // packed or aligned (private enum) json_validator: Option, // None when no JSON Schema supplied + validation_plan: Arc, // compiled value-domain constraints (ADR-012 §3) endian: Endian, // parsed from the root struct's "endian" bast_doc: Value, // retained for sequential_reader/read_field root_name: String, // the selected $defs entry @@ -175,6 +192,7 @@ impl AlkTypeEngine { pub fn validate_json(&self, instance: &Value) -> Result<(), AlkTypeError>; // D-BAST-007 pub fn is_valid_json(&self, instance: &Value) -> bool; // D-BAST-007 pub fn validate_bytes(&self, buffer: &[u8]) -> Result<(), AlkTypeError>; // D-BAST-006 + pub fn validation_plan(&self) -> &Arc; // compiled constraints (ADR-012 §3) } ``` @@ -273,16 +291,22 @@ The expensive work happens once at schema load time: 1. Parse the BAST document into the typed tree (`BastDoc::new`). 2. Parse the root struct's `"endian"` annotation. -3. Compute the layout (`LayoutBuilder` for packed, `OffsetMap` for +3. Compile the `ValidationPlan` — the value-domain constraint tree, + with eager `$ref` resolution. Its compile walk rejects cyclic `$ref` + graphs with a clean `Schema` error *before* the layout computation: + the layout walkers' struct/union recursion has no cycle guard, so + ordering the plan compile first is what keeps a self-referential + (malicious or accidental) document a handleable error, not a stack + overflow. +4. Compute the layout (`LayoutBuilder` for packed, `OffsetMap` for aligned). -4. If `json_schema` is `Some`, build the standard +5. If `json_schema` is `Some`, build the standard `jsonschema::Validator` via `validation::build_validator`. The result is an `AlkTypeEngine` that can be used for repeated -operations. The BAST-native validator is not pre-built — it is a -recursive walker that runs on the materialized `Value` at access time, -re-using the `BastDoc` (re-parsed on demand from the retained -`bast_doc`). +operations. The validation half is pre-built: the engine holds an +`Arc` and walks it per buffer without re-touching the +BAST document (ADR-012 §3). ### Access time: `engine.validate_bytes(&[u8])` @@ -297,10 +321,13 @@ sequence (D-BAST-006): → dispatch then recurse; `Record` → object of key/value entries). The read phase reuses the existing data-access functions and returns `AlkTypeError::Access` (with field paths) on read failures. -2. **Validate the `Value`.** The materialized `Value` is passed to - `bast_validation::validate_value(&doc, &value)`, producing +2. **Validate the `Value` against the `ValidationPlan`.** The + materialized `Value` is walked against the compiled constraint tree + (`engine.validation_plan().validate(&value)`), producing `AlkTypeError::Validation` on the first violated value-domain - constraint. + constraint. This is the ADR-012 §3 end state: the only per-buffer + schema-touching step is the materialize half (the bytes must be + decoded against the tree); the validation half is plan-fast. Mode dispatch: @@ -308,7 +335,7 @@ Mode dispatch: - **Aligned mode** — uses the `OffsetMap` to read fields at their computed offsets. -Both modes produce the same `Value` form; the BAST-native validator is +Both modes produce the same `Value` form; the validation plan is mode-agnostic. ### Access time: `engine.validate_json(&Value)` / `engine.is_valid_json(&Value)` @@ -379,9 +406,10 @@ then access the binary buffer. | Decision | ADR | Summary | |----------|-----|---------| -| Two-validator model (BAST-native + standard jsonschema) | [ADR-VAL-SPLIT](decisions/val-split-two-validator-model.md) | `validate_bytes` uses the BAST-native validator; `validate_json` uses a standard `jsonschema::Validator` from a consumer-provided JSON Schema; D-BAST-006/007/009 | +| Two-validator model (BAST-native + standard jsonschema) | [ADR-VAL-SPLIT](decisions/val-split-two-validator-model.md) | `validate_bytes` uses the compiled `ValidationPlan`; `validate_json` uses a standard `jsonschema::Validator` from a consumer-provided JSON Schema; D-BAST-006/007/009 | | Error handling and validation strategy | [ADR-004](decisions/004-error-handling-validation-strategy.md) | `AlkTypeError` enum; load-time build, access-time check; field-path-carrying errors; jsonschema `ValidationError` wrapping | | Generalized validation — `validate_bytes` | [ADR-010](decisions/010-generalized-validation-validate-bytes.md) | Single-call binary-buffer validation (materialize `Value` from bytes, then validate); two methods on one struct, not a trait | +| Compiled `ValidationPlan` | [ADR-012](decisions/012-plan-fingerprinting-and-m1-closure.md) | The value-domain constraint tree is compiled once at `compile` (eager `$ref` resolution, cycle rejection) and walked per buffer; `Hash + Eq` + `fingerprint()`; retires the interpretive `BastDoc` walk | | BAST format | [ADR-BAST](decisions/bast-bast-format.md) | The BAST document is the complete binary-format spec (layout + value constraints) | ## Open Questions @@ -397,15 +425,19 @@ see [builder.md](builder.md). — the normative validation model - [ADR-VAL-SPLIT](decisions/val-split-two-validator-model.md) — the two-validator decision +- [ADR-012](decisions/012-plan-fingerprinting-and-m1-closure.md) — the + `ValidationPlan` decision (§3) - [ADR-004](decisions/004-error-handling-validation-strategy.md) — error handling and validation strategy - [ADR-010](decisions/010-generalized-validation-validate-bytes.md) — `validate_bytes` (the collapsed two-step dance) - [schema-layer.md](schema-layer.md) — the BAST parser that the - BAST-native validator walks + plan compiler consumes - [data-access.md](data-access.md) — read/write functions and the - materializer that produce the `Value` the validator checks + materializer that produce the `Value` the plan checks +- [`src/validation_plan.rs`](../../src/validation_plan.rs) — the + compiled `ValidationPlan` implementation - [`src/bast_validation.rs`](../../src/bast_validation.rs) — the - BAST-native validator implementation + one-shot wrapper (`validate_value`) and shared error helpers - [`src/validation.rs`](../../src/validation.rs) — the `build_validator` helper \ No newline at end of file diff --git a/docs/plans/030-compiled-forms.md b/docs/plans/030-compiled-forms.md index 5df9a2d..b013d83 100644 --- a/docs/plans/030-compiled-forms.md +++ b/docs/plans/030-compiled-forms.md @@ -1,7 +1,7 @@ --- status: in-progress created: 2026-08-19 -last_updated: 2026-08-20 +last_updated: 2026-08-31 adr: ADR-011, ADR-012 --- @@ -566,7 +566,36 @@ are trait derives, `fingerprint` is an inherent method). --- -## Phase 7 — `ValidationPlan` (ADR-012 §3) +## Phase 7 — `ValidationPlan` (ADR-012 §3) — **DONE (2026-08-31)** + +> **Status: implemented.** The design session ran and the shape landed +> in `src/validation_plan.rs`. Summary of what was decided and built +> (full detail in ADR-012 §3a): +> +> - **Shape:** `ValidationPlan { root: ValidNode }` — a constraint tree +> with one arm per value-domain check (`Int`/`I64`/`Uint`/`U64`/ +> `Float`/`Bool`/`Str`/`Bytes`/`Enum`/`Struct`/`Union`/`Array`/ +> `Record`), `ValidField { name, node }`, `ValidVariant { key, node }`. +> `maxLength` baked into leaf nodes from the owning field at compile +> time. Unions compile to variant nodes only (the declared union +> fields are validated via the variant walk, matching the interpretive +> arm's dispatch-on-`__discriminator` semantics). +> - **`compile` signature:** `compile(&BastDoc) -> Result AlkTypeError>` (the plan table's `&str` param was vestigial). +> - **`bast_validation`:** the interpretive walker is *retired* +> (deleted, not just bypassed); `validate_value` survives as a +> compile-once-per-call wrapper over the plan (one-shot/diagnostic +> use); shared error helper retained. +> - **Engine:** `Arc` built at `compile` in both modes; +> new accessor `validation_plan()`. `validate_bytes` walks the plan. +> **Bonus:** `ValidationPlan::compile` runs before the layout build and +> serves as the engine's cyclic-`$ref` gate (the layout walkers have no +> cycle guard; a cyclic doc used to be a stack-overflow hazard there — +> now a clean `Schema` error, see ADR-012 Consequences). +> - **Remaining phase-7 bench work** (a `validate_bytes`-stream bench +> in alktty) moves with the bench work into phase 8; a spot check +> during development measured plan-validate at ~0.2 µs/call vs ~0.6 +> µs for the compile-per-call one-shot it replaced. **Goal:** Retire the interpretive `bast_validation` walk. Introduce a `ValidationPlan` — a compile-once-walk-many compiled form over the diff --git a/src/bast_validation.rs b/src/bast_validation.rs index 93d9e48..0fdc38a 100644 --- a/src/bast_validation.rs +++ b/src/bast_validation.rs @@ -1,44 +1,34 @@ -//! BAST-native validator — the `validate_bytes` validation step. +//! BAST-native validation — the `validate_bytes` validation step, +//! now a thin entry over the compiled [`crate::validation_plan:: +//! ValidationPlan`] (ADR-012 §3, 0.3.0 phase 7). //! -//! A recursive walker over the BAST typed tree ([`crate::bast::BastDoc`]/ -//! [`crate::bast::BastType`]) that checks the value-domain constraints the -//! materializer ([`crate::materialize`]) does NOT check. This replaces the -//! v0.1.0 `jsonschema` custom-keyword validators on the bytes path -//! (D-BAST-006) with a flat match — no factories, no trait objects, no -//! sub-validator pre-computation. +//! ## History //! -//! ## What the materializer already guarantees +//! Through 0.2.0 this module *was* the validator: a recursive walker +//! over the BAST typed tree ([`crate::bast::BastDoc`]) that resolved +//! `$ref`s lazily and rebuilt path strings per node, on every +//! `validate_bytes` call. That interpretive walk is the same class of +//! per-buffer cost review #004 measured on the read path (400x), so +//! ADR-012 §3 (per review #005 M3) committed the `ValidationPlan`: the +//! constraint tree (enum allowed-sets, integer ranges, `maxLength` caps, +//! union variant keys, array counts, record value types) is compiled +//! once at engine-compile time and walked per buffer with no `BastDoc` +//! touch. The interpretive walker's checks moved verbatim into the plan +//! (see [`crate::validation_plan`]'s constraint table); the last +//! interpretive consumers of this module are gone. //! -//! By construction, a materialized [`serde_json::Value`] is structurally -//! correct: all declared fields are present, types are correct, bounds -//! are checked (via [`crate::data_access`]), UTF-8 is valid, the boolean -//! byte is 0 or 1, and the union discriminator is in the mapping. The -//! validator only needs to enforce the **value-domain constraints -//! expressed in the BAST document** — the ones the materializer can't -//! see from the bytes alone. +//! ## What remains here //! -//! ## Constraint table -//! -//! See [bast-format.md §Validation Model](../../docs/architecture/bast-format.md#validation-model). -//! The arms of the `validate_typeref` walker implement the table: -//! -//! | Constraint | Validator arm | -//! |---|---| -//! | Integer range (Int8..Uint64) | `validate_int`/`validate_uint` | -//! | Int64/Uint64 (full range) | `validate_int64`/`validate_uint64` | -//! | Float finiteness (Float32/64) | `validate_float` | -//! | String `maxLength` (byte length) | `check_string` reads [`BastField::max_length`](crate::bast::BastField::max_length) | -//! | Bytes `maxLength` (array length) | `check_bytes` | -//! | Enum index bounds | `validate_enum` — **fixes the v0.1.0 dead constraint** | -//! | Union variant dispatch | `validate_union` reads `__discriminator`, resolves the variant, recurses | -//! | Struct fields | `validate_struct` walks `fields`, requires each declared field present, recurses | -//! | Array count | `validate_array` checks `arr.len() == count` and recurses per element | -//! | Record values | `validate_record` recurses into each value's `values` type | -//! | Boolean | `validate_bool` (materializer already rejects non-0/1 bytes) | +//! - [`validate_value`] — the public one-shot entry (`compile` + +//! `validate`), retained for API compatibility and for callers holding +//! a `BastDoc` without an engine. The engine does not use it per +//! buffer; it holds a compiled plan (ADR-012 §3). +//! - `validation_err` / `DISCRIMINATOR_KEY` — shared helpers, also used +//! by the plan walk. //! //! ## Error payload (D-BAST-009) //! -//! The bytes path no longer touches `jsonschema` for validation, but the +//! The bytes path does not touch `jsonschema` for validation, but the //! error variant retains the `jsonschema::ValidationError<'static>` type //! for uniformity with the `validate_json` path. Errors are constructed //! via [`jsonschema::ValidationError::custom`] so consumers handle one @@ -46,324 +36,40 @@ //! //! ## Untrusted input //! -//! Every walk path returns [`AlkTypeError::Schema`] on a malformed BAST -//! document, never `panic!`/`unreachable!`/`unwrap` (AGENTS.md §3). The -//! typed tree is parsed once by [`BastDoc::new`](crate::bast::BastDoc::new); -//! lazy `$ref` resolution via [`BastDoc::resolve_typeref`] surfaces a -//! `Schema` error for dangling refs. +//! Malformed BAST documents surface as [`AlkTypeError::Schema`] from +//! [`ValidationPlan::compile`](crate::validation_plan::ValidationPlan::compile) +//! — including cyclic `$ref` graphs, which the interpretive walker could +//! not reject (it would overflow the stack) and now cannot reach. -use crate::bast::{ - BastArray, BastDefKind, BastDoc, BastEnum, BastField, BastRecord, BastStruct, BastType, - BastUnion, -}; +use crate::bast::BastDoc; use crate::error::AlkTypeError; -use crate::schema::AlkTypeKind; use serde_json::Value; -const DISCRIMINATOR_KEY: &str = "__discriminator"; +pub(crate) const DISCRIMINATOR_KEY: &str = "__discriminator"; /// Validate a materialized `value` against the BAST root type. /// -/// This is the `validate_bytes` validation step: after -/// [`crate::materialize::materialize_packed`]/ -/// [`crate::materialize::materialize_aligned`] produces a `Value` tree -/// from the bytes, this walker enforces the value-domain constraints -/// expressed in the BAST document. The materializer already guarantees -/// structural correctness; the validator only checks what the bytes -/// alone can't tell you (integer ranges, `maxLength`, -/// enum index bounds, union variant constraints). +/// This is the convenience one-shot form: it compiles a +/// [`ValidationPlan`](crate::validation_plan::ValidationPlan) from `doc` +/// and validates `value` against it. Use this when you hold a BAST +/// document and a materialized `Value` but no engine. In the engine's +/// hot path (`AlkTypeEngine::validate_bytes`, ADR-010/ADR-012 §3), the +/// plan is compiled once and re-walked per buffer without +/// re-touching the document. /// /// Returns `Err(AlkTypeError::Validation(...))` on the first violated /// constraint, or `Err(AlkTypeError::Schema(...))` if the BAST document -/// is malformed (a dangling `$ref`, a missing `values` array, etc.). +/// is malformed (a dangling `$ref`, a cyclic `$ref`, a missing `values` +/// array, etc.). pub fn validate_value(doc: &BastDoc<'_>, value: &Value) -> Result<(), AlkTypeError> { - let root_def = doc.root_def(); - match root_def.kind() { - BastDefKind::Struct(s) => validate_struct(doc, s, value, ""), - BastDefKind::Union(u) => validate_union(doc, u, value, ""), - BastDefKind::Enum(e) => validate_enum(value, "", e), - } -} - -/// Recursively validate `value` against `ty`, resolving `$ref`s lazily. -/// -/// `field` is the owning field (for `maxLength`/annotations) — `None` for -/// synthetic contexts (array elements, record values, the root). The -/// validator reads [`BastField::max_length`] only for `String`/`Bytes` -/// arms, so passing a synthetic field (which has `max_length == None`) is -/// correct for those contexts. -fn validate_typeref( - doc: &BastDoc<'_>, - ty: &BastType<'_>, - field: Option<&BastField<'_>>, - value: &Value, - path: &str, -) -> Result<(), AlkTypeError> { - let resolved = doc.resolve_typeref(ty)?; - match &resolved { - BastType::Primitive(AlkTypeKind::Int8) => validate_int(value, path, -128, 127), - BastType::Primitive(AlkTypeKind::Int16) => validate_int(value, path, -32768, 32767), - BastType::Primitive(AlkTypeKind::Int32) => { - validate_int(value, path, -2147483648, 2147483647) - } - BastType::Primitive(AlkTypeKind::Int64) => validate_int64(value, path), - BastType::Primitive(AlkTypeKind::Uint8) => validate_uint(value, path, 255), - BastType::Primitive(AlkTypeKind::Uint16) => validate_uint(value, path, 65535), - BastType::Primitive(AlkTypeKind::Uint32) => validate_uint(value, path, 4294967295), - BastType::Primitive(AlkTypeKind::Uint64) => validate_uint64(value, path), - BastType::Primitive(AlkTypeKind::Float32) => validate_float(value, path), - BastType::Primitive(AlkTypeKind::Float64) => validate_float(value, path), - BastType::Primitive(AlkTypeKind::Boolean) => validate_bool(value, path), - BastType::Primitive(AlkTypeKind::String) => { - check_string(value, path, field.and_then(|f| f.max_length())) - } - BastType::Primitive(AlkTypeKind::Bytes) => { - check_bytes(value, path, field.and_then(|f| f.max_length())) - } - BastType::Enum(e) => validate_enum(value, path, e), - BastType::Struct(s) => validate_struct(doc, s, value, path), - BastType::Union(u) => validate_union(doc, u, value, path), - BastType::Array(a) => validate_array(doc, a, value, path), - BastType::Record(r) => validate_record(doc, r, value, path), - BastType::Ref(_) => Err(AlkTypeError::Schema(format!( - "bast_validation: unresolved $ref at {path} (resolve_typeref should have deref'd it)" - ))), - BastType::Primitive(k) => Err(AlkTypeError::Schema(format!( - "bast_validation: unsupported primitive kind {k} at {path}" - ))), - } -} - -fn validate_int(value: &Value, path: &str, min: i64, max: i64) -> Result<(), AlkTypeError> { - let n = value - .as_i64() - .ok_or_else(|| validation_err(path, "expected an integer"))?; - if n < min || n > max { - return Err(validation_err( - path, - format!("integer {n} out of range [{min}, {max}]"), - )); - } - Ok(()) -} - -fn validate_int64(value: &Value, path: &str) -> Result<(), AlkTypeError> { - if value.as_i64().is_none() { - return Err(validation_err(path, "expected an i64 integer")); - } - Ok(()) -} - -fn validate_uint(value: &Value, path: &str, max: u64) -> Result<(), AlkTypeError> { - let n = value - .as_u64() - .ok_or_else(|| validation_err(path, "expected a non-negative integer"))?; - if n > max { - return Err(validation_err(path, format!("integer {n} exceeds {max}"))); - } - Ok(()) -} - -fn validate_uint64(value: &Value, path: &str) -> Result<(), AlkTypeError> { - if value.as_u64().is_none() { - return Err(validation_err(path, "expected a u64 integer")); - } - Ok(()) -} - -fn validate_float(value: &Value, path: &str) -> Result<(), AlkTypeError> { - match value { - Value::Number(n) => { - let f = n - .as_f64() - .ok_or_else(|| validation_err(path, "expected a number"))?; - if !f.is_finite() { - return Err(validation_err(path, "expected a finite number")); - } - Ok(()) - } - _ => Err(validation_err(path, "expected a number")), - } -} - -fn validate_bool(value: &Value, path: &str) -> Result<(), AlkTypeError> { - match value { - Value::Bool(_) => Ok(()), - _ => Err(validation_err(path, "expected a boolean")), - } -} - -fn check_string(value: &Value, path: &str, max_length: Option) -> Result<(), AlkTypeError> { - let s = value - .as_str() - .ok_or_else(|| validation_err(path, "expected a string"))?; - if let Some(max) = max_length { - if s.len() > max { - return Err(validation_err( - path, - format!("string byte length {} exceeds maxLength {max}", s.len()), - )); - } - } - Ok(()) -} - -fn check_bytes(value: &Value, path: &str, max_length: Option) -> Result<(), AlkTypeError> { - let arr = value - .as_array() - .ok_or_else(|| validation_err(path, "expected an array of u8 for bytes"))?; - if let Some(max) = max_length { - if arr.len() > max { - return Err(validation_err( - path, - format!("bytes array length {} exceeds maxLength {max}", arr.len()), - )); - } - } - for (i, entry) in arr.iter().enumerate() { - let n = entry.as_u64().ok_or_else(|| { - validation_err( - path, - format!("bytes array entry {i} is not a non-negative integer"), - ) - })?; - if n > 255 { - return Err(validation_err( - path, - format!("bytes array entry {i} = {n} is not a u8 (0..=255)"), - )); - } - } - Ok(()) -} - -/// Enum validation on the bytes path: the materializer emits a numeric -/// index (`Value::Number`), and the constraint is that the index is -/// within the `values` array bounds (`0..len-1`). -/// -/// This is the **fix for the v0.1.0 dead constraint**: the built-in -/// `enum` keyword checked string membership, but the materializer emitted -/// a numeric index that never matched — so out-of-bounds enum indices -/// silently passed. The BAST-native validator checks the index bounds -/// directly. -fn validate_enum(value: &Value, path: &str, enum_def: &BastEnum<'_>) -> Result<(), AlkTypeError> { - let idx = value - .as_u64() - .ok_or_else(|| validation_err(path, "expected a non-negative integer enum index"))?; - let len = enum_def.values().len() as u64; - if idx >= len { - return Err(validation_err( - path, - format!("enum index {idx} out of bounds (values has {len} entries)"), - )); - } - Ok(()) -} - -fn validate_struct( - doc: &BastDoc<'_>, - struct_def: &BastStruct<'_>, - value: &Value, - path: &str, -) -> Result<(), AlkTypeError> { - let obj = value - .as_object() - .ok_or_else(|| validation_err(path, "expected an object"))?; - for field in struct_def.fields() { - let name = field.name(); - let field_path = if path.is_empty() { - name.to_string() - } else { - format!("{path}.{name}") - }; - let field_value = obj.get(name).ok_or_else(|| { - validation_err(&field_path, format!("missing field {name:?}")) - })?; - validate_typeref(doc, field.ty(), Some(field), field_value, &field_path)?; - } - Ok(()) -} - -/// Union validation: read `__discriminator`, look up the variant -/// [`BastType`] in the union's `mapping`, and recurse into the variant. -/// -/// This recovers OQ-008 per-variant constraint enforcement (e.g. -/// `maxLength` on a `bytes` field inside a variant struct) without -/// custom keywords — the recursion walks the variant's BAST definition -/// and enforces every field constraint it declares. -fn validate_union( - doc: &BastDoc<'_>, - union_def: &BastUnion<'_>, - value: &Value, - path: &str, -) -> Result<(), AlkTypeError> { - let obj = value - .as_object() - .ok_or_else(|| validation_err(path, "expected an object for union"))?; - let disc = obj.get(DISCRIMINATOR_KEY).ok_or_else(|| { - validation_err(path, "union instance is missing the '__discriminator' field") - })?; - let key = match disc { - Value::String(s) => s.clone(), - Value::Number(n) => n.to_string(), - _ => { - return Err(validation_err( - path, - "union '__discriminator' must be a string or number", - )); - } - }; - let variant_ty = union_def.variant_for(&key).ok_or_else(|| { - validation_err(path, format!("union discriminator value '{key}' not in mapping")) - })?; - validate_typeref(doc, variant_ty, None, value, path) -} - -fn validate_array( - doc: &BastDoc<'_>, - array_def: &BastArray<'_>, - value: &Value, - path: &str, -) -> Result<(), AlkTypeError> { - let arr = value - .as_array() - .ok_or_else(|| validation_err(path, "expected an array"))?; - let count = array_def.count(); - if arr.len() != count { - return Err(validation_err( - path, - format!("array length {} does not match declared count {count}", arr.len()), - )); - } - let element_ty = array_def.element(); - for (i, item) in arr.iter().enumerate() { - let item_path = format!("{path}[{i}]"); - validate_typeref(doc, element_ty, None, item, &item_path)?; - } - Ok(()) -} - -fn validate_record( - doc: &BastDoc<'_>, - record_def: &BastRecord<'_>, - value: &Value, - path: &str, -) -> Result<(), AlkTypeError> { - let obj = value - .as_object() - .ok_or_else(|| validation_err(path, "expected an object for record"))?; - let values_ty = record_def.values(); - for (k, v) in obj.iter() { - let entry_path = format!("{path}[{k}]"); - validate_typeref(doc, values_ty, None, v, &entry_path)?; - } - Ok(()) + let plan = crate::validation_plan::ValidationPlan::compile(doc)?; + plan.validate(value) } /// Construct a `Validation` error from a path + reason string. The /// payload is a `jsonschema::ValidationError::custom` so the variant /// type stays uniform with the `validate_json` path (D-BAST-009). -fn validation_err(path: &str, reason: impl Into) -> AlkTypeError { +pub(crate) fn validation_err(path: &str, reason: impl Into) -> AlkTypeError { let msg = if path.is_empty() { reason.into() } else { @@ -375,6 +81,7 @@ fn validation_err(path: &str, reason: impl Into) -> AlkTypeError { #[cfg(test)] mod tests { use super::*; + use crate::materialize::materialize_packed; use serde_json::json; fn doc_from<'a>(root: &'a Value, name: &'a str) -> BastDoc<'a> { @@ -395,10 +102,15 @@ mod tests { doc: &BastDoc<'_>, buffer: &[u8], ) -> Result<(), AlkTypeError> { - let value = crate::materialize::materialize_packed(doc, buffer)?; + let value = materialize_packed(doc, buffer)?; validate_value(doc, &value) } + // The tests below are the *parity* suite: they drive validation + // end-to-end (materialize -> validate_value) through the public API + // exactly as the interpretive walker's tests did, so any behavioral + // change in the compiled plan shows up here. + // ----- Integer ranges -------------------------------------------------- #[test] @@ -744,4 +456,21 @@ mod tests { assert!(validate_value(&d, &json!({"flag": false})).is_ok()); assert!(validate_value(&d, &json!({"flag": "yes"})).is_err()); } + + // ----- Cyclic schema: the new compile-time guard ---------------------- + + #[test] + fn cyclic_bast_doc_rejected_with_schema_error() { + let root = json!({ + "$defs": { + "S": { "kind": "struct", "fields": [ + { "name": "me", "kind": { "$ref": "#/$defs/S" } } + ] } + } + }); + let d = doc_from(&root, "S"); + let err = validate_value(&d, &json!({})).unwrap_err(); + assert!(matches!(err, AlkTypeError::Schema(_)), "got {err:?}"); + assert!(err.to_string().contains("cyclic"), "got {err:?}"); + } } \ No newline at end of file diff --git a/src/engine.rs b/src/engine.rs index fa2a13a..0522649 100644 --- a/src/engine.rs +++ b/src/engine.rs @@ -17,7 +17,6 @@ //! [overview.md](../../docs/architecture/overview.md). use crate::bast::{BastDefKind, BastDoc, BastStruct, BastType}; -use crate::bast_validation; use crate::data_access; use crate::error::AlkTypeError; use crate::layout_builder::LayoutBuilder; @@ -26,8 +25,10 @@ use crate::offset_map::OffsetMap; use crate::schema::{AlkTypeKind, Endian, VariableEncoding}; use crate::sequential_reader::{FieldValue, SequentialReader}; use crate::validation; +use crate::validation_plan::ValidationPlan; use serde_json::Value; use std::fmt; +use std::sync::Arc; /// The layout mode selected at engine construction time. #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -71,7 +72,7 @@ enum Layout { /// /// Validation is split (D-BAST-006, D-BAST-007): /// - [`AlkTypeEngine::validate_bytes`] uses the BAST-native validator -/// ([`bast_validation`]) — no `jsonschema` involvement, no external + /// ([`crate::bast_validation`]) — no `jsonschema` involvement, no external /// JSON Schema required. /// - [`AlkTypeEngine::validate_json`] / [`AlkTypeEngine::is_valid_json`] /// use a standard `jsonschema::Validator` compiled at construction @@ -81,6 +82,7 @@ enum Layout { pub struct AlkTypeEngine { layout: Layout, json_validator: Option, + validation_plan: Arc, endian: Endian, bast_doc: Value, root_name: String, @@ -139,6 +141,13 @@ impl AlkTypeEngine { } }; let endian = struct_node.endian(); + // The compiled value-domain constraint tree (ADR-012 §3) — built + // once here, walked per buffer by `validate_bytes`. It is also the + // engine's reference-graph gate: its compile walk rejects cyclic + // `$ref` graphs (with a clean `Schema` error) before the layout + // builders below, whose struct/union recursion has no cycle guard + // and would otherwise recurse unboundedly on such a document. + let validation_plan = Arc::new(ValidationPlan::compile(&doc)?); let layout = match mode { LayoutMode::Packed => { let builder = LayoutBuilder::new(bast_doc, root_name)?; @@ -156,6 +165,7 @@ impl AlkTypeEngine { Ok(Self { layout, json_validator, + validation_plan, endian, bast_doc: bast_doc.clone(), root_name: root_name.to_string(), @@ -256,17 +266,22 @@ impl AlkTypeEngine { /// Validate a binary buffer against the schema by materializing a /// `serde_json::Value` tree from the bytes (walking the layout engine) - /// and then validating that `Value` against the BAST-native validator - /// — a recursive walker over the BAST type tree that checks the - /// value-domain constraints the materializer does not (integer - /// ranges, `maxLength`, enum index bounds, union - /// variant constraints). Decided in D-BAST-006; see + /// and then validating that `Value` against the compiled + /// [`ValidationPlan`] — the engine's value-domain constraint tree + /// (integer ranges, `maxLength`, enum index bounds, union variant + /// constraints), built once at [`AlkTypeEngine::compile`] from the + /// BAST document (ADR-010; ADR-012 §3). Decided in D-BAST-006; see /// [bast-format.md §Validation Model](../../docs/architecture/bast-format.md#validation-model). /// + /// The materialize half is the only per-buffer schema-touching step + /// (the bytes must be decoded against the tree); the validation half + /// walks the compiled plan — no `$ref` re-resolution, no schema + /// re-parse per buffer. + /// /// Dispatches on the engine's layout mode: packed mode walks /// sequentially from offset 0; aligned mode reads at offsets from /// the `OffsetMap`. Both produce the same `Value` form; the - /// BAST-native validator is mode-agnostic. + /// validation plan is mode-agnostic. /// /// # Errors /// @@ -288,7 +303,22 @@ impl AlkTypeEngine { materialize::materialize_aligned(&doc, buffer, offset_map)? } }; - bast_validation::validate_value(&doc, &value) + self.validation_plan.validate(&value) + } + + /// Access the compiled [`ValidationPlan`] (ADR-012 §3). + /// + /// The plan is the engine's value-domain constraint tree — built once + /// at [`AlkTypeEngine::compile`] from the BAST document, shared via + /// `Arc`, and walked by [`AlkTypeEngine::validate_bytes`] per buffer + /// without touching the BAST document. Exposed for consumers that + /// want to validate their *own* materialized `Value` trees + /// (e.g. one produced by an external reader) against the same + /// constraints, or that want the plan's + /// [`fingerprint`](ValidationPlan::fingerprint) for caching or + /// schema handshakes. + pub fn validation_plan(&self) -> &Arc { + &self.validation_plan } /// Read a field from a buffer at its computed offset (aligned mode). @@ -1287,6 +1317,85 @@ mod tests { assert!(engine.validate_bytes(&buf).is_ok()); } + // ----- ValidationPlan / validate_bytes (ADR-012 §3, phase 7) ---------- + + #[test] + fn validation_plan_accessor_returns_compiled_plan() { + let doc = uint32_struct_bast(); + let engine = + AlkTypeEngine::compile(&doc, "S", LayoutMode::Packed, None).expect("compile"); + let plan = engine.validation_plan(); + assert!(plan.validate(&json!({"id": 42})).is_ok()); + assert!(plan.validate(&json!({"id": -1})).is_err()); + // Same schema compiled twice produces equal plans (fingerprint + // contract), so the accessor reflects the compile-time build. + let engine2 = + AlkTypeEngine::compile(&doc, "S", LayoutMode::Packed, None).expect("compile"); + assert_eq!(plan, engine2.validation_plan()); + assert_eq!(plan.fingerprint(), engine2.validation_plan().fingerprint()); + } + + #[test] + fn validate_bytes_enforces_value_domain_via_plan_in_both_modes() { + let doc = json!({ + "$defs": { "S": { "kind": "struct", "endian": "little", "fields": [ + { "name": "status", "kind": { "$ref": "#/$defs/Status" } } + ] }, + "Status": { "kind": "enum", "values": ["Ok", "Err"] } } + }); + // Packed: enum index 5 is out of bounds for the 2-value enum. + let engine = AlkTypeEngine::compile(&doc, "S", LayoutMode::Packed, None).expect("compile"); + let err = engine.validate_bytes(&5u32.to_le_bytes()).unwrap_err(); + assert!(matches!(err, AlkTypeError::Validation(_)), "got {err:?}"); + // Aligned: same constraint, offset read path (the enum leaf sits + // at offset 0 — no alignment padding for a lone u32-width leaf). + let engine = + AlkTypeEngine::compile(&doc, "S", LayoutMode::Aligned, None).expect("compile"); + let mut buf = [0u8; 8]; + buf[0..4].copy_from_slice(&5u32.to_le_bytes()); + let err = engine.validate_bytes(&buf).unwrap_err(); + assert!(matches!(err, AlkTypeError::Validation(_)), "got {err:?}"); + // Valid index passes in both modes. + let engine = AlkTypeEngine::compile(&doc, "S", LayoutMode::Packed, None).expect("compile"); + assert!(engine.validate_bytes(&1u32.to_le_bytes()).is_ok()); + let engine = + AlkTypeEngine::compile(&doc, "S", LayoutMode::Aligned, None).expect("compile"); + let mut buf = [0u8; 8]; + buf[0..4].copy_from_slice(&1u32.to_le_bytes()); + assert!(engine.validate_bytes(&buf).is_ok()); + } + + #[test] + fn compile_rejects_cyclic_ref_graph_with_schema_error() { + let doc = json!({ + "$defs": { + "A": { "kind": "struct", "fields": [ + { "name": "next", "kind": { "$ref": "#/$defs/B" } } + ] }, + "B": { "kind": "struct", "fields": [ + { "name": "back", "kind": { "$ref": "#/$defs/A" } } + ] } + } + }); + let err = AlkTypeEngine::compile(&doc, "A", LayoutMode::Packed, None).unwrap_err(); + assert!(matches!(err, AlkTypeError::Schema(_)), "got {err:?}"); + assert!(err.to_string().contains("cyclic"), "got {err:?}"); + let err = AlkTypeEngine::compile(&doc, "A", LayoutMode::Aligned, None).unwrap_err(); + assert!(matches!(err, AlkTypeError::Schema(_)), "got {err:?}"); + } + + #[test] + fn validation_plan_is_send_sync_shared() { + let doc = uint32_struct_bast(); + let engine = + AlkTypeEngine::compile(&doc, "S", LayoutMode::Packed, None).expect("compile"); + fn assert_send_sync(_: &T) {} + assert_send_sync(engine.validation_plan()); + let shared:std::sync::Arc<_> = engine.validation_plan().clone(); + let handle = std::thread::spawn(move || shared.validate(&json!({"id": 1})).is_ok()); + assert!(handle.join().expect("join")); + } + // ----- validate_json / is_valid_json tests (step 6, D-BAST-007) ----- fn uint32_struct_bast() -> Value { diff --git a/src/lib.rs b/src/lib.rs index 6c42ca3..839cc5d 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -19,13 +19,17 @@ //! offsets, zero-copy for fixed-size types. //! - **TUnion dispatch** ([`tunion`]): Byte-offset and field-name //! discriminator dispatch over `BastUnion`. -//! - **Validation** ([`validation`], [`bast_validation`]): two -//! validators for two paths. `bast_validation` is the BAST-native -//! validator for `validate_bytes` — a recursive walker over the BAST -//! type tree (D-BAST-006). `validation` builds a standard -//! `jsonschema::Validator` from a consumer-provided JSON Schema for -//! `validate_json` / `is_valid_json` (D-BAST-007) — no custom keywords, -//! no BAST involvement. +//! - **Validation** ([`validation`], [`bast_validation`], +//! [`validation_plan`]): two validators for two paths. +//! `validation_plan` is the compiled `ValidationPlan` — a +//! compile-once-walk-many constraint tree over the BAST document's +//! value-domain constraints, walked by `validate_bytes` per buffer +//! without re-touching the document (ADR-012 §3). `bast_validation` +//! hosts the one-shot `validate_value` wrapper over the plan. +//! `validation` builds a standard `jsonschema::Validator` from a +//! consumer-provided JSON Schema for `validate_json` / +//! `is_valid_json` (D-BAST-007) — no custom keywords, no BAST +//! involvement. //! - **Builder** ([`builder`]): Fluent Rust API for constructing BAST //! documents (binary layout) and standard JSON Schemas (JSON //! validation) at runtime, producing `serde_json::Value` (ADR-009, @@ -52,6 +56,7 @@ pub mod schema; pub mod sequential_reader; pub mod tunion; pub mod validation; +pub mod validation_plan; pub use bast_meta::BAST_META_SCHEMA; pub use bast::{ @@ -67,3 +72,4 @@ pub use schema::{Endian, AlkTypeKind, VariableEncoding}; pub use sequential_reader::{FieldValue, SequentialReader}; pub use tunion::UnionDispatch; pub use validation::build_validator; +pub use validation_plan::{ValidField, ValidNode, ValidVariant, ValidationPlan}; diff --git a/src/validation_plan.rs b/src/validation_plan.rs new file mode 100644 index 0000000..d546c73 --- /dev/null +++ b/src/validation_plan.rs @@ -0,0 +1,1133 @@ +//! `ValidationPlan` — the compiled validation form over a BAST document's +//! value-domain constraints (ADR-012 §3, 0.3.0 phase 7). +//! +//! This is the value-domain analog of the read-side compiled forms: where +//! [`crate::sequential_reader`] walks a byte-position plan and +//! [`crate::offset_map::OffsetMap`] walks precomputed offsets, this +//! module walks a **constraint tree** computed once at compile time. The +//! per-buffer interpretive cost — re-resolving `$ref`s (deep-cloning +//! inline types per field per buffer via +//! [`crate::bast::BastDoc::resolve_typeref`]) and rebuilding field-path +//! strings on every node — is paid once at [`ValidationPlan::compile`] +//! instead of per buffer. +//! +//! ## What the plan checks +//! +//! Exactly the constraint set the interpretive +//! [`crate::bast_validation`] walker enforced — nothing more, nothing +//! less (behavioral parity): +//! +//! | Constraint | Plan node | +//! |-------------------------------|----------------------| +//! | Integer range (Int8..Int32) | [`ValidNode::Int`] | +//! | Int64 full range | [`ValidNode::I64`] | +//! | Integer range (Uint8..Uint32) | [`ValidNode::Uint`] | +//! | Uint64 full range | [`ValidNode::U64`] | +//! | Float finiteness | [`ValidNode::Float`] | +//! | Boolean | [`ValidNode::Bool`] | +//! | String `maxLength` (bytes) | [`ValidNode::Str`] | +//! | Bytes `maxLength` (u8 array) | [`ValidNode::Bytes`] | +//! | Enum index bounds | [`ValidNode::Enum`] | +//! | Union variant dispatch | [`ValidNode::Union`] | +//! | Struct fields | [`ValidNode::Struct`]| +//! | Array count | [`ValidNode::Array`] | +//! | Record values | [`ValidNode::Record`]| +//! +//! Fields with no declared `maxLength` compile to `max_len: None`, which +//! the walk accepts without a length check — parity with the interpretive +//! walk. `maxLength` is baked into the leaf node from the *owning field* +//! at compile time; it never crosses a `$ref` (a `$ref` always lands on a +//! struct/union/enum `$defs` entry, never a primitive, so the interpretive +//! walk could never consult it through one either). +//! +//! The union's declared `fields` (shared fields for field-name +//! discriminators) are validated only as part of the *variant* walk: the +//! materialized `__discriminator` dispatch selects a variant and the +//! whole object is validated against the variant node, mirroring +//! [`crate::bast_validation`]'s union arm, which recursed into the +//! variant alone. +//! +//! ## Error rendering +//! +//! Paths are assembled from a segment stack only when a constraint +//! fails, so the happy path performs **zero allocations and zero path +//! formatting**. Error messages are byte-identical in format to the +//! interpretive walker's (`"a.b[0].c: reason"`, same wording per +//! constraint), so consumers matching on +//! payloads see no behavioral change. Record keys render as +//! `{path}[{key}]` and array elements as `{path}[{i}]`, matching the +//! interpretive walker. +//! +//! ## Untrusted input +//! +//! Compile is the trust boundary: a malformed or cyclically-referencing +//! BAST document surfaces as [`AlkTypeError::Schema`] at `compile` time +//! (a depth cap plus a definition-level cycle set protect the stack — +//! the interpretive walker had neither), never a panic (AGENTS.md §3). +//! The walk itself works over compiled nodes and materialized `Value`s +//! only — both sides are already-parsed data. + +use crate::bast::{BastDefKind, BastDoc, BastStruct, BastType, BastUnion}; +use crate::bast_validation::{validation_err, DISCRIMINATOR_KEY}; +use crate::error::AlkTypeError; +use crate::schema::AlkTypeKind; +use serde_json::Value; +use std::collections::BTreeSet; +use std::fmt::Write as _; + +/// Maximum recursion depth for the compile walk. Bounds deeply-nested +/// (or cyclically-referencing) adversarial schemas; the interpretive +/// walker had no guard and could overflow the stack on such input. +const MAX_COMPILE_DEPTH: usize = 128; + +/// The compiled value-domain constraint tree (ADR-012 §3). +/// +/// Built once via [`ValidationPlan::compile`]; walked per buffer by +/// [`ValidationPlan::validate`]. Immutable, owned, `Send + Sync` (the +/// engine shares it via `Arc`), with a [`PartialEq`]/[`Hash`]-stable +/// structural form and a [`Self::fingerprint`] for downstream plan-cache +/// and handshake uses. +/// +/// The fingerprint contract (ADR-012 §1, §3): two validation plans with +/// equal hashes accept/reject identical `(bytes)` inputs identically. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub struct ValidationPlan { + root: ValidNode, +} + +impl ValidationPlan { + /// Compile the value-domain constraints of a BAST document into a + /// plan. + /// + /// Walks the typed tree once, eagerly resolving every `$ref` (the + /// interpretive walk resolved them lazily per buffer). The root may + /// be any of the three `$defs` composite kinds — struct, union, or + /// enum — matching [`crate::bast_validation`]'s entry conditions. + /// + /// # Errors + /// + /// Returns [`AlkTypeError::Schema`] if the document is malformed + /// (dangling `$ref`, unsupported node) or if the reference graph + /// nests deeper than the compile depth cap (128) or contains a + /// reference cycle (a compile-time rejection of self-recursive + /// schemas). + pub fn compile(doc: &BastDoc<'_>) -> Result { + let root_def = doc.root_def(); + let mut seen = BTreeSet::new(); + let root = match root_def.kind() { + BastDefKind::Struct(s) => compile_struct(doc, s, "", 0, &mut seen)?, + BastDefKind::Union(u) => compile_union(doc, u, "", 0, &mut seen)?, + BastDefKind::Enum(e) => compile_enum_node(e)?, + }; + Ok(ValidationPlan { root }) + } + + /// Validate a materialized `Value` tree against this plan. + /// + /// This is the per-buffer walk that replaces the interpretive + /// `BastDoc` recursion: dispatch on the compiled node (no `$ref` + /// resolution, no schema re-parse), with a segment stack for error + /// paths (rendered only on failure). + /// + /// # Errors + /// + /// Returns [`AlkTypeError::Validation`] with the same message formats + /// as the interpretive walk on the first violated constraint. + pub fn validate(&self, value: &Value) -> Result<(), AlkTypeError> { + let mut path: Vec> = Vec::new(); + validate_node(&self.root, value, &mut path) + } + + /// A stable 64-bit fingerprint of this plan's accept/reject behavior. + /// + /// Two plans with the same fingerprint accept and reject identical + /// `(bytes)` inputs identically (the fingerprint contract). Stable + /// within a crate version; see ADR-012 §1. + pub fn fingerprint(&self) -> u64 { + use std::hash::{Hash, Hasher}; + let mut h = std::hash::DefaultHasher::new(); + self.hash(&mut h); + h.finish() + } + + /// The compiled root constraint node (diagnostic access). + pub fn root(&self) -> &ValidNode { + &self.root + } +} + +/// A compiled constraint node — one arm per value-domain check. +/// +/// Public for diagnostics: consumers can inspect the compiled +/// constraints via [`ValidationPlan::root`]. Construction is only +/// possible through [`ValidationPlan::compile`]. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub enum ValidNode { + /// Signed integer in `[min, max]`. Compiled from `Int8`/`Int16`/ + /// `Int32`. + Int { + min: i64, + max: i64, + }, + /// `Int64` — any `i64` value. + I64, + /// Unsigned integer in `[0, max]`. Compiled from `Uint8`/`Uint16`/ + /// `Uint32`. + Uint { + max: u64, + }, + /// `Uint64` — any `u64` value. + U64, + /// `Float32`/`Float64` — finiteness check. + Float, + /// `Boolean`. + Bool, + /// `String` with an optional byte-length cap. + Str { + max_len: Option, + }, + /// `Bytes` — array of u8 with an optional length cap. + Bytes { + max_len: Option, + }, + /// `Enum` — index in `[0, count)`. + Enum { + count: u64, + }, + /// `Struct` — every declared field must be present and valid. + Struct { + fields: Vec, + }, + /// `Union` — the materialized `__discriminator` selects a variant; + /// the whole union instance (including `__discriminator`) is then + /// validated against the variant node. + Union { + variants: Vec, + }, + /// `Array` — length must equal the declared count; each element + /// validated against the element node. + Array { + count: usize, + element: Box, + }, + /// `Record` — each value validated against the values node. + Record { + values: Box, + }, +} + +/// One declared struct field with its compiled constraint node. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub struct ValidField { + name: String, + node: ValidNode, +} + +impl ValidField { + /// The field's declared name. + pub fn name(&self) -> &str { + &self.name + } + + /// The field's compiled constraint node (`maxLength` already baked + /// in, where applicable). + pub fn node(&self) -> &ValidNode { + &self.node + } +} + +/// One union mapping entry with its compiled variant node. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub struct ValidVariant { + key: String, + node: ValidNode, +} + +impl ValidVariant { + /// The discriminator value this variant answers to (the mapping key). + pub fn key(&self) -> &str { + &self.key + } + + /// The variant's compiled constraint node. + pub fn node(&self) -> &ValidNode { + &self.node + } +} + +fn compile_enum_node(e: &crate::bast::BastEnum<'_>) -> Result { + let count = u64::try_from(e.values().len()).map_err(|_| { + AlkTypeError::Schema("bast_validation: enum values len overflows u64".to_string()) + })?; + Ok(ValidNode::Enum { count }) +} + +fn depth_err(path: &str) -> AlkTypeError { + AlkTypeError::Schema(format!( + "validation_plan: compile depth exceeded {MAX_COMPILE_DEPTH} at {path} \ + (cyclic $ref or adversarially deep nesting)" + )) +} + +fn cycle_err(name: &str, path: &str) -> AlkTypeError { + AlkTypeError::Schema(format!( + "validation_plan: cyclic $ref through {name:?} at {path}" + )) +} + +/// Compile a possibly-`$ref` type node to its concrete constraint node, +/// eagerly. `max_len` is the owning field's `maxLength` (baked into +/// `String`/`Bytes` leaves; never crosses a `$ref`, whose target is +/// always a struct/union/enum `$defs` entry). +fn compile_typeref( + doc: &BastDoc<'_>, + ty: &BastType<'_>, + max_len: Option, + path: &str, + depth: usize, + seen: &mut BTreeSet, +) -> Result { + if depth > MAX_COMPILE_DEPTH { + return Err(depth_err(path)); + } + match ty { + BastType::Ref(r) => { + let name = r.name(); + if !seen.insert(name.to_string()) { + return Err(cycle_err(name, path)); + } + let def = doc.resolve_ref(r)?; + let node = match def.kind() { + BastDefKind::Struct(s) => compile_struct(doc, s, path, depth + 1, seen)?, + BastDefKind::Union(u) => compile_union(doc, u, path, depth + 1, seen)?, + BastDefKind::Enum(e) => compile_enum_node(e)?, + }; + seen.remove(name); + Ok(node) + } + other => compile_resolved(doc, other, max_len, path, depth, seen), + } +} + +/// Compile a dereferenced type node (the `Ref` case never reaches here). +fn compile_resolved( + doc: &BastDoc<'_>, + ty: &BastType<'_>, + max_len: Option, + path: &str, + depth: usize, + seen: &mut BTreeSet, +) -> Result { + match ty { + BastType::Primitive(AlkTypeKind::Int8) => { + Ok(ValidNode::Int { min: i8::MIN as i64, max: i8::MAX as i64 }) + } + BastType::Primitive(AlkTypeKind::Int16) => { + Ok(ValidNode::Int { min: i16::MIN as i64, max: i16::MAX as i64 }) + } + BastType::Primitive(AlkTypeKind::Int32) => { + Ok(ValidNode::Int { min: i32::MIN as i64, max: i32::MAX as i64 }) + } + BastType::Primitive(AlkTypeKind::Int64) => Ok(ValidNode::I64), + BastType::Primitive(AlkTypeKind::Uint8) => Ok(ValidNode::Uint { max: u8::MAX as u64 }), + BastType::Primitive(AlkTypeKind::Uint16) => Ok(ValidNode::Uint { max: u16::MAX as u64 }), + BastType::Primitive(AlkTypeKind::Uint32) => Ok(ValidNode::Uint { max: u32::MAX as u64 }), + BastType::Primitive(AlkTypeKind::Uint64) => Ok(ValidNode::U64), + BastType::Primitive(AlkTypeKind::Float32 | AlkTypeKind::Float64) => Ok(ValidNode::Float), + BastType::Primitive(AlkTypeKind::Boolean) => Ok(ValidNode::Bool), + BastType::Primitive(AlkTypeKind::String) => Ok(ValidNode::Str { max_len }), + BastType::Primitive(AlkTypeKind::Bytes) => Ok(ValidNode::Bytes { max_len }), + BastType::Primitive(other) => Err(AlkTypeError::Schema(format!( + "bast_validation: unsupported primitive kind {other} at {path}" + ))), + BastType::Enum(e) => compile_enum_node(e), + BastType::Struct(s) => compile_struct(doc, s, path, depth + 1, seen), + BastType::Union(u) => compile_union(doc, u, path, depth + 1, seen), + BastType::Array(a) => { + if depth > MAX_COMPILE_DEPTH { + return Err(depth_err(path)); + } + let element = + compile_typeref(doc, a.element(), None, path, depth + 1, seen)?; + Ok(ValidNode::Array { count: a.count(), element: Box::new(element) }) + } + BastType::Record(r) => { + if depth > MAX_COMPILE_DEPTH { + return Err(depth_err(path)); + } + let values = + compile_typeref(doc, r.values(), None, path, depth + 1, seen)?; + Ok(ValidNode::Record { values: Box::new(values) }) + } + BastType::Ref(_) => Err(AlkTypeError::Schema(format!( + "bast_validation: unresolved $ref at {path} \ + (should have been deref'd by compile_typeref)" + ))), + } +} + +fn compile_struct( + doc: &BastDoc<'_>, + s: &BastStruct<'_>, + path: &str, + depth: usize, + seen: &mut BTreeSet, +) -> Result { + if depth > MAX_COMPILE_DEPTH { + return Err(depth_err(path)); + } + let mut fields = Vec::with_capacity(s.fields().len()); + for field in s.fields() { + let field_path = if path.is_empty() { + field.name().to_string() + } else { + format!("{path}.{}", field.name()) + }; + let node = compile_typeref( + doc, + field.ty(), + field.max_length(), + &field_path, + depth + 1, + seen, + )?; + fields.push(ValidField { + name: field.name().to_string(), + node, + }); + } + Ok(ValidNode::Struct { fields }) +} + +fn compile_union( + doc: &BastDoc<'_>, + u: &BastUnion<'_>, + path: &str, + depth: usize, + seen: &mut BTreeSet, +) -> Result { + if depth > MAX_COMPILE_DEPTH { + return Err(depth_err(path)); + } + let mut variants = Vec::with_capacity(u.mapping().len()); + for (key, ty) in u.mapping() { + let variant_path = format!("{path}.mapping[{key}]"); + let node = compile_typeref(doc, ty, None, &variant_path, depth + 1, seen)?; + variants.push(ValidVariant { + key: (*key).to_string(), + node, + }); + } + Ok(ValidNode::Union { variants }) +} + +/// One already-rendered path segment in the error-path stack. +enum PathSeg<'a> { + Field(&'a str), + Index(usize), + Key(&'a str), +} + +impl PathSeg<'_> { + fn render(&self, out: &mut String) { + match self { + PathSeg::Field(name) => { + if !out.is_empty() { + out.push('.'); + } + out.push_str(name); + } + PathSeg::Index(i) => { + let _ = write!(out, "[{i}]"); + } + PathSeg::Key(k) => { + let _ = write!(out, "[{k}]"); + } + } + } +} + +fn render_path(path: &[PathSeg<'_>]) -> String { + let mut out = String::new(); + for seg in path { + seg.render(&mut out); + } + out +} + +fn err_at(path: &[PathSeg<'_>], reason: impl std::fmt::Display) -> AlkTypeError { + let rendered = render_path(path); + validation_err(&rendered, reason.to_string()) +} + +fn validate_node<'a>( + node: &'a ValidNode, + value: &'a Value, + path: &mut Vec>, +) -> Result<(), AlkTypeError> { + match node { + ValidNode::Int { min, max } => { + let n = value + .as_i64() + .ok_or_else(|| err_at(path, "expected an integer"))?; + if n < *min || n > *max { + return Err(err_at( + path, + format_args!("integer {n} out of range [{min}, {max}]"), + )); + } + Ok(()) + } + ValidNode::I64 => { + if value.as_i64().is_none() { + return Err(err_at(path, "expected an i64 integer")); + } + Ok(()) + } + ValidNode::Uint { max } => { + let n = value + .as_u64() + .ok_or_else(|| err_at(path, "expected a non-negative integer"))?; + if n > *max { + return Err(err_at(path, format_args!("integer {n} exceeds {max}"))); + } + Ok(()) + } + ValidNode::U64 => { + if value.as_u64().is_none() { + return Err(err_at(path, "expected a u64 integer")); + } + Ok(()) + } + ValidNode::Float => match value { + Value::Number(n) => { + let f = n + .as_f64() + .ok_or_else(|| err_at(path, "expected a number"))?; + if !f.is_finite() { + return Err(err_at(path, "expected a finite number")); + } + Ok(()) + } + _ => Err(err_at(path, "expected a number")), + }, + ValidNode::Bool => match value { + Value::Bool(_) => Ok(()), + _ => Err(err_at(path, "expected a boolean")), + }, + ValidNode::Str { max_len } => { + let s = value + .as_str() + .ok_or_else(|| err_at(path, "expected a string"))?; + if let Some(max) = max_len { + if s.len() > *max { + return Err(err_at( + path, + format_args!("string byte length {} exceeds maxLength {max}", s.len()), + )); + } + } + Ok(()) + } + ValidNode::Bytes { max_len } => { + let arr = value + .as_array() + .ok_or_else(|| err_at(path, "expected an array of u8 for bytes"))?; + if let Some(max) = max_len { + if arr.len() > *max { + return Err(err_at( + path, + format_args!("bytes array length {} exceeds maxLength {max}", arr.len()), + )); + } + } + for (i, entry) in arr.iter().enumerate() { + let n = entry.as_u64().ok_or_else(|| { + err_at( + path, + format_args!("bytes array entry {i} is not a non-negative integer"), + ) + })?; + if n > 255 { + return Err(err_at( + path, + format_args!("bytes array entry {i} = {n} is not a u8 (0..=255)"), + )); + } + } + Ok(()) + } + ValidNode::Enum { count } => { + let idx = value + .as_u64() + .ok_or_else(|| err_at(path, "expected a non-negative integer enum index"))?; + if idx >= *count { + return Err(err_at( + path, + format_args!( + "enum index {idx} out of bounds (values has {count} entries)" + ), + )); + } + Ok(()) + } + ValidNode::Struct { fields } => { + let obj = value + .as_object() + .ok_or_else(|| err_at(path, "expected an object"))?; + for field in fields { + path.push(PathSeg::Field(field.name())); + let field_value = obj + .get(field.name()) + .ok_or_else(|| err_at(path, format_args!("missing field {:?}", field.name())))?; + let res = validate_node(field.node(), field_value, path); + path.pop(); + res?; + } + Ok(()) + } + ValidNode::Union { variants } => { + let obj = value + .as_object() + .ok_or_else(|| err_at(path, "expected an object for union"))?; + let disc = obj + .get(DISCRIMINATOR_KEY) + .ok_or_else(|| { + err_at(path, "union instance is missing the '__discriminator' field") + })?; + let key = match disc { + Value::String(s) => s.as_str(), + Value::Number(n) => { + return validate_union_numeric(variants, n, value, path); + } + _ => { + return Err(err_at( + path, + "union '__discriminator' must be a string or number", + )); + } + }; + let variant = variants + .iter() + .find(|v| v.key() == key) + .ok_or_else(|| { + err_at( + path, + format_args!( + "union discriminator value '{key}' not in mapping" + ), + ) + })?; + validate_node(variant.node(), value, path) + } + ValidNode::Array { count, element } => { + let arr = value + .as_array() + .ok_or_else(|| err_at(path, "expected an array"))?; + if arr.len() != *count { + return Err(err_at( + path, + format_args!( + "array length {} does not match declared count {count}", + arr.len() + ), + )); + } + for (i, item) in arr.iter().enumerate() { + path.push(PathSeg::Index(i)); + let res = validate_node(element, item, path); + path.pop(); + res?; + } + Ok(()) + } + ValidNode::Record { values } => { + let obj = value + .as_object() + .ok_or_else(|| err_at(path, "expected an object for record"))?; + for (k, v) in obj.iter() { + path.push(PathSeg::Key(k)); + let res = validate_node(values, v, path); + path.pop(); + res?; + } + Ok(()) + } + } +} + +/// Validate a numeric (`Value::Number`) discriminator. The interpretive +/// walker stringified the number with `n.to_string()` and looked the key +/// up in the mapping; this does the equivalent without allocating by +/// first matching the small integer forms directly, then falling back to +/// the string form (values are always stringified integers for byte +/// discriminators, so a direct integer match covers every real case). +fn validate_union_numeric<'p>( + variants: &'p [ValidVariant], + n: &serde_json::Number, + whole: &'p Value, + path: &mut Vec>, +) -> Result<(), AlkTypeError> { + let variant = if let Some(i) = n.as_u64() { + variants.iter().find(|v| v.key() == i.to_string()) + } else if let Some(i) = n.as_i64() { + variants.iter().find(|v| v.key() == i.to_string()) + } else { + variants.iter().find(|v| v.key() == n.to_string()) + }; + match variant { + Some(v) => validate_node(v.node(), whole, path), + None => { + let key = n.to_string(); + Err(err_at( + path, + format_args!("union discriminator value '{key}' not in mapping"), + )) + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + fn doc_from<'a>(root: &'a Value, name: &'a str) -> BastDoc<'a> { + BastDoc::new(root, name).expect("bast doc") + } + + fn plan_for(root: &Value, name: &str) -> ValidationPlan { + let d = doc_from(root, name); + ValidationPlan::compile(&d).expect("compile") + } + + // ----- Parity with the interpretive walker ---------------------------- + + #[test] + fn parity_uint32_range() { + let root = json!({ + "$defs": { "S": { "kind": "struct", "fields": [ + { "name": "id", "kind": "uint32" } + ] } } + }); + let plan = plan_for(&root, "S"); + assert!( + plan.validate(&json!({"id": -1})).is_err(), + "negative must fail" + ); + assert!( + plan.validate(&json!({"id": 4294967296u64})).is_err(), + "above u32 must fail" + ); + assert!(plan.validate(&json!({"id": 0})).is_ok()); + assert!(plan.validate(&json!({"id": 4294967295u64})).is_ok()); + assert!(plan.validate(&json!({"id": "x"})).is_err()); + } + + #[test] + fn parity_int8_range() { + let root = json!({ + "$defs": { "S": { "kind": "struct", "fields": [ + { "name": "v", "kind": "int8" } + ] } } + }); + let plan = plan_for(&root, "S"); + assert!(plan.validate(&json!({"v": 0})).is_ok()); + assert!(plan.validate(&json!({"v": 127})).is_ok()); + assert!(plan.validate(&json!({"v": -128})).is_ok()); + assert!(plan.validate(&json!({"v": 128})).is_err()); + assert!(plan.validate(&json!({"v": -129})).is_err()); + } + + #[test] + fn parity_int64_uint64() { + let root = json!({ + "$defs": { "S": { "kind": "struct", "fields": [ + { "name": "a", "kind": "int64" }, + { "name": "b", "kind": "uint64" } + ] } } + }); + let plan = plan_for(&root, "S"); + assert!(plan.validate(&json!({"a": 0, "b": 0})).is_ok()); + assert!(plan + .validate(&json!({ + "a": 9223372036854775807i64, + "b": 18446744073709551615u64 + })) + .is_ok()); + assert!(plan.validate(&json!({"a": "x", "b": 0})).is_err()); + assert!(plan.validate(&json!({"a": 0, "b": -1})).is_err()); + } + + #[test] + fn parity_string_max_length() { + let root = json!({ + "$defs": { "S": { "kind": "struct", "fields": [ + { "name": "name", "kind": "string", "maxLength": 3 } + ] } } + }); + let plan = plan_for(&root, "S"); + assert!(plan.validate(&json!({"name": "hi"})).is_ok()); + let err = plan.validate(&json!({"name": "hello"})).unwrap_err(); + assert!( + matches!(err, AlkTypeError::Validation(_)), + "got {err:?}" + ); + } + + #[test] + fn parity_bytes_max_length_and_u8_entries() { + let root = json!({ + "$defs": { "S": { "kind": "struct", "fields": [ + { "name": "blob", "kind": "bytes", "maxLength": 2 } + ] } } + }); + let plan = plan_for(&root, "S"); + assert!(plan.validate(&json!({"blob": [0xAA, 0xBB]})).is_ok()); + assert!(plan.validate(&json!({"blob": [1, 2, 3]})).is_err()); + + let root = json!({ + "$defs": { "S": { "kind": "struct", "fields": [ + { "name": "blob", "kind": "bytes" } + ] } } + }); + let plan = plan_for(&root, "S"); + assert!(plan.validate(&json!({"blob": [65, 256]})).is_err()); + assert!(plan.validate(&json!({"blob": [65, "x"]})).is_err()); + assert!(plan.validate(&json!({"blob": [65, 255]})).is_ok()); + } + + #[test] + fn parity_enum_index_bounds() { + let root = json!({ + "$defs": { + "S": { "kind": "struct", "fields": [ + { "name": "status", "kind": { "$ref": "#/$defs/StatusCode" } } + ] }, + "StatusCode": { "kind": "enum", "values": ["Ok", "Error", "Pending"] } + } + }); + let plan = plan_for(&root, "S"); + assert!(plan.validate(&json!({"status": 0})).is_ok()); + assert!(plan.validate(&json!({"status": 2})).is_ok()); + assert!(plan.validate(&json!({"status": 3})).is_err()); + assert!(plan.validate(&json!({"status": "Ok"})).is_err()); + } + + #[test] + fn parity_union_dispatch_byte_disc() { + let root = json!({ + "$defs": { + "U": { + "kind": "union", + "discriminator": { "kind": "byte", "offset": 0, "type": "uint8" }, + "mapping": { + "5": { "kind": "struct", "fields": [ { "name": "id", "kind": "uint32" } ] } + } + } + } + }); + let plan = plan_for(&root, "U"); + assert!(plan + .validate(&json!({"__discriminator": 5, "id": 42})) + .is_ok()); + assert!(plan + .validate(&json!({"__discriminator": 5, "id": -1})) + .is_err()); + assert!(plan + .validate(&json!({"__discriminator": 99, "id": 42})) + .is_err()); + assert!(plan.validate(&json!({"id": 42})).is_err()); + assert!(plan.validate(&json!("not-object")).is_err()); + } + + #[test] + fn parity_union_dispatch_field_disc() { + let root = json!({ + "$defs": { + "Event": { + "kind": "union", + "discriminator": { "kind": "field", "name": "type" }, + "fields": [ { "name": "type", "kind": "string" } ], + "mapping": { "data": { "$ref": "#/$defs/Data" } } + }, + "Data": { "kind": "struct", "fields": [ + { "name": "blob", "kind": "bytes", "maxLength": 1 } + ] } + } + }); + let plan = plan_for(&root, "Event"); + let ok = json!({ + "__discriminator": "data", + "type": "data", + "blob": [1] + }); + assert!(plan.validate(&ok).is_ok()); + let bad = json!({ + "__discriminator": "data", + "type": "data", + "blob": [1, 2] + }); + assert!(plan.validate(&bad).is_err()); + } + + #[test] + fn parity_array_count_and_struct_elements() { + let root = json!({ + "$defs": { + "S": { "kind": "struct", "fields": [ + { "name": "pts", "kind": { + "kind": "array", "element": "uint16", "count": 2 + } } + ] } + } + }); + let plan = plan_for(&root, "S"); + assert!(plan.validate(&json!({"pts": [1, 2]})).is_ok()); + assert!(plan.validate(&json!({"pts": [1, 2, 3]})).is_err()); + assert!(plan.validate(&json!({"pts": [1]})).is_err()); + assert!(plan.validate(&json!({"pts": "x"})).is_err()); + + let root = json!({ + "$defs": { + "S": { "kind": "struct", "fields": [ + { "name": "points", "kind": { + "kind": "array", + "element": { "$ref": "#/$defs/Point" }, + "count": 2 + } } + ] }, + "Point": { "kind": "struct", "fields": [ + { "name": "x", "kind": "uint16" }, + { "name": "y", "kind": "uint16" } + ] } + } + }); + let plan = plan_for(&root, "S"); + assert!(plan.validate(&json!({"pts": [{"x": 0, "y": 0}]})).is_err()); + let good = json!({"points": [{"x": 1, "y": 2}, {"x": 3, "y": 4}]}); + assert!(plan.validate(&good).is_ok()); + let bad = json!({"points": [{"x": 1, "y": 2}, {"x": 3}]}); + assert!(plan.validate(&bad).is_err()); + } + + #[test] + fn parity_record_values() { + let root = json!({ + "$defs": { + "S": { "kind": "struct", "fields": [ + { "name": "counts", "kind": { "kind": "record", "values": "uint32" } } + ] } + } + }); + let plan = plan_for(&root, "S"); + assert!(plan.validate(&json!({"counts": {"a": 1, "b": 2}})).is_ok()); + assert!(plan.validate(&json!({"counts": {"a": -1}})).is_err()); + assert!(plan.validate(&json!({"counts": "not-object"})).is_err()); + } + + #[test] + fn parity_struct_missing_field() { + let root = json!({ + "$defs": { "S": { "kind": "struct", "fields": [ + { "name": "id", "kind": "uint32" }, + { "name": "flag", "kind": "uint8" } + ] } } + }); + let plan = plan_for(&root, "S"); + assert!(plan.validate(&json!({"id": 1, "flag": 0})).is_ok()); + assert!(plan.validate(&json!({"id": 1})).is_err()); + } + + #[test] + fn parity_float_and_bool() { + let root = json!({ + "$defs": { "S": { "kind": "struct", "fields": [ + { "name": "f", "kind": "float32" }, + { "name": "flag", "kind": "bool" } + ] } } + }); + let plan = plan_for(&root, "S"); + assert!(plan.validate(&json!({"f": 3.5, "flag": true})).is_ok()); + assert!(plan.validate(&json!({"f": 0, "flag": false})).is_ok()); + assert!(plan.validate(&json!({"f": "x", "flag": false})).is_err()); + assert!(plan.validate(&json!({"f": 0.0, "flag": "yes"})).is_err()); + } + + // ----- Error message format parity (paths) ---------------------------- + + #[test] + fn error_paths_match_interpretive_format() { + let root = json!({ + "$defs": { + "S": { "kind": "struct", "fields": [ + { "name": "inner", "kind": { "kind": "struct", "fields": [ + { "name": "blob", "kind": "bytes", "maxLength": 1 } + ] } } + ] } + } + }); + let plan = plan_for(&root, "S"); + let err = plan + .validate(&json!({"inner": {"blob": [1, 2]}})) + .unwrap_err(); + let msg = err.to_string(); + assert!( + msg.contains("inner.blob"), + "expected nested path in {msg}" + ); + assert!( + msg.contains("maxLength"), + "expected maxLength wording in {msg}" + ); + + // Array element path rendering: `pts[0]`. + let root = json!({ + "$defs": { + "S": { "kind": "struct", "fields": [ + { "name": "pts", "kind": { + "kind": "array", "element": "uint16", "count": 1 + } } + ] } + } + }); + let plan = plan_for(&root, "S"); + let err = plan.validate(&json!({"pts": [65536]})).unwrap_err(); + let msg = err.to_string(); + assert!(msg.contains("pts[0]"), "expected array index in {msg}"); + + // Record key rendering: `counts[a]`. + let root = json!({ + "$defs": { + "S": { "kind": "struct", "fields": [ + { "name": "counts", "kind": { "kind": "record", "values": "uint32" } } + ] } + } + }); + let plan = plan_for(&root, "S"); + let err = plan.validate(&json!({"counts": {"a": -1}})).unwrap_err(); + let msg = err.to_string(); + assert!(msg.contains("counts[a]"), "expected record key in {msg}"); + } + + // ----- Fingerprint contract ------------------------------------------- + + #[test] + fn fingerprint_stable_across_compiles_and_sensitive_to_changes() { + let root = json!({ + "$defs": { "S": { "kind": "struct", "fields": [ + { "name": "id", "kind": "uint32" } + ] } } + }); + let p1 = plan_for(&root, "S"); + let p2 = plan_for(&root, "S"); + assert_eq!(p1, p2); + assert_eq!(p1.fingerprint(), p2.fingerprint()); + + let changed = json!({ + "$defs": { "S": { "kind": "struct", "fields": [ + { "name": "id", "kind": "uint8" } + ] } } + }); + let p3 = plan_for(&changed, "S"); + assert_ne!(p1.fingerprint(), p3.fingerprint()); + } + + // ----- Untrusted schema: cycles, depth, malformed --------------------- + + #[test] + fn cyclic_schema_rejected_at_compile() { + let root = json!({ + "$defs": { + "A": { "kind": "struct", "fields": [ + { "name": "next", "kind": { "$ref": "#/$defs/B" } } + ] }, + "B": { "kind": "struct", "fields": [ + { "name": "back", "kind": { "$ref": "#/$defs/A" } } + ] } + } + }); + let d = doc_from(&root, "A"); + let err = ValidationPlan::compile(&d).unwrap_err(); + assert!(matches!(err, AlkTypeError::Schema(_)), "got {err:?}"); + } + + #[test] + fn deep_nesting_rejected_not_stack_overflow() { + // Build a 200-deep inline struct nesting. Meta-schema would reject + // this, but the plan compiler must survive it on its own (the + // engine's meta-schema validation runs before compile in + // `AlkTypeEngine::compile`, but `ValidationPlan::compile` is + // public and must be safe standalone). + let mut fields = json!([{ "name": "v", "kind": "uint8" }]); + for _ in 0..200 { + fields = json!([{ "name": "n", "kind": { "kind": "struct", "fields": fields } }]); + } + let root = json!({ "$defs": { "S": { "kind": "struct", "fields": fields } } }); + let d = doc_from(&root, "S"); + let err = ValidationPlan::compile(&d).unwrap_err(); + assert!(matches!(err, AlkTypeError::Schema(_)), "got {err:?}"); + assert!( + err.to_string().contains("depth exceeded"), + "expected depth error, got {err:?}" + ); + } + + #[test] + fn self_referential_ref_chain_rejected() { + let root = json!({ + "$defs": { + "S": { "kind": "struct", "fields": [ + { "name": "me", "kind": { "$ref": "#/$defs/S" } } + ] } + } + }); + let d = doc_from(&root, "S"); + let err = ValidationPlan::compile(&d).unwrap_err(); + assert!(matches!(err, AlkTypeError::Schema(_)), "got {err:?}"); + assert!( + err.to_string().contains("cyclic"), + "expected cycle error, got {err:?}" + ); + } + + #[test] + fn shared_refs_compile_without_false_cycle() { + // Diamond references: two fields referencing the same def is not + // a cycle. The `seen` set is path-scoped, so this must compile. + let root = json!({ + "$defs": { + "S": { "kind": "struct", "fields": [ + { "name": "a", "kind": { "$ref": "#/$defs/Point" } }, + { "name": "b", "kind": { "$ref": "#/$defs/Point" } } + ] }, + "Point": { "kind": "struct", "fields": [ + { "name": "x", "kind": "uint16" } + ] } + } + }); + let d = doc_from(&root, "S"); + let plan = ValidationPlan::compile(&d).expect("diamond refs compile"); + assert!(plan + .validate(&json!({"a": {"x": 1}, "b": {"x": 2}})) + .is_ok()); + } + + #[test] + fn enum_root_compiles() { + let root = json!({ + "$defs": { "Status": { "kind": "enum", "values": ["On", "Off"] } } + }); + let plan = plan_for(&root, "Status"); + assert!(plan.validate(&json!(1)).is_ok()); + assert!(plan.validate(&json!(2)).is_err()); + assert!(plan.validate(&json!("On")).is_err()); + } + + #[test] + fn send_sync_bounds_hold() { + fn assert_send_sync() {} + assert_send_sync::(); + assert_send_sync::(); + } +} \ No newline at end of file