Add BAST-native validator POC for validate_bytes
Implements the D-BAST-006 validation model as a self-contained module (`src/bast_poc.rs`) that does not touch the production schema / materializer / validator paths. Reuses only `data_access`, `AlkTypeError`, and `Endian` — the BAST parser, packed-mode materializer, and native validator are all from scratch, which is the point: prove the model works end-to-end before refactoring. The POC confirms the hypothesis from the pivot doc: a recursive walker over the BAST type tree fully replaces the 19 custom keyword validators on the `validate_bytes` path, recovers the OQ-008 union variant dispatch, and fixes the enum-membership dead constraint on the bytes path — all without `jsonschema` custom keywords and without an external JSON Schema. Coverage (20 tests, all passing): - basic: chunk header, int8/uint32 ranges, string/bytes maxLength, timestamp, bool, short buffer - hard: union byte-offset disc + maxLength inside variant (OQ-008), union field-name disc + maxLength inside variant, enum index-bounds fix, nested struct→union→struct, array of structs with count, record, untrusted-schema error-not-panic Verification: - cargo test --release (416 tests: 330 lib + 86 integration) - cargo clippy --all-targets -- -D warnings - cargo build --target wasm32-unknown-unknown --release - cargo doc --no-deps Findings and verdict recorded in docs/research/bast-pivot.md §"POC Result — BAST-native validator".
This commit is contained in:
1 parent
19f8162f1a
commit
f371fe4e94
3 files changed
+1605
No files matched your search
@@ -1165,3 +1165,153 @@ readers/writers from BAST documents for these crates.
|
||||
- `/workspace/@alkimiadev/typebox-rs/src/codegen/` — Rust/TypeScript codegen from schemas (reference architecture)
|
||||
- `/workspace/alknet-typedef-poc/tests/sftp_roundtrip_test.rs` — SFTP POC proving byte-identical output (to be replicated with BAST)
|
||||
- `/workspace/@alkdev/alkcall/src/channels/wire.rs` — hand-rolled chunk header (target for BAST replacement)
|
||||
|
||||
## POC Result — BAST-native validator
|
||||
|
||||
**Status: succeeded.** The POC is on branch `bast-validator-poc` in
|
||||
`src/bast_poc.rs` (20 tests, all passing; full crate suite — 416 tests —
|
||||
green; `cargo clippy --all-targets -- -D warnings` clean;
|
||||
`cargo build --target wasm32-unknown-unknown --release` clean).
|
||||
|
||||
The POC implements the `validate_bytes` validation model from
|
||||
[D-BAST-006](#d-bast-006-validate_bytes-validation-model) as a
|
||||
self-contained module that does **not** touch the production schema /
|
||||
materializer / validator paths. It reuses only `data_access` (read
|
||||
primitives), `AlkTypeError` (error type), and `Endian`. The BAST
|
||||
document parser, a packed-mode materializer, and the BAST-native
|
||||
validator are all implemented from scratch — that is the point: prove
|
||||
the model works end-to-end before refactoring the production code.
|
||||
|
||||
### What the POC proves
|
||||
|
||||
The hypothesis from the [POC section](#poc-bast-native-validator-for-validate_bytes)
|
||||
is confirmed: a recursive walker over the BAST type tree fully
|
||||
replaces the 19 custom keyword validators on the `validate_bytes` path,
|
||||
including the OQ-008 union variant dispatch, and fixes the
|
||||
enum-membership dead constraint — all without `jsonschema` custom
|
||||
keywords and without an external JSON Schema.
|
||||
|
||||
The hard cases that were the actual de-risking targets all pass:
|
||||
|
||||
- **Union byte-offset discriminator + `maxLength` inside a variant
|
||||
(OQ-008).** `union_byte_disc_max_length_inside_variant_enforced`
|
||||
materializes a union with two `$ref` variants, dispatches on a
|
||||
byte-offset `uint8` discriminator, and enforces `maxLength` on a
|
||||
`bytes` field inside the selected variant. The validator reads
|
||||
`__discriminator`, looks up the variant's BAST definition, and
|
||||
recurses — same behavior as the current `UnionValidator`'s
|
||||
per-variant sub-validators, but with no `jsonschema` involvement.
|
||||
- **Union field-name discriminator + `maxLength` inside a variant.**
|
||||
`union_field_disc_max_length_inside_variant_enforced` covers the
|
||||
typedef.ts-style discriminator (a length-prefixed string field
|
||||
selects the variant). Same recursion model.
|
||||
- **Enum index-bounds fix.** `enum_index_out_of_bounds_rejected`
|
||||
exercises the constraint that is **broken in the current engine**
|
||||
(the built-in `enum` keyword checks string membership; the
|
||||
materializer emits `Value::Number(index)`, which never matches — a
|
||||
dead constraint). The BAST-native validator checks the materialized
|
||||
index against the `values` array bounds (0..len-1), which is the
|
||||
correct validation for a binary enum encoded as an index. Net
|
||||
improvement, not a regression.
|
||||
- **Nested struct wrapping a union wrapping a struct.**
|
||||
`nested_struct_with_union_variant` confirms the recursion composes
|
||||
through multiple type layers.
|
||||
- **Arrays of fixed-size structs with `count`.**
|
||||
`array_of_structs_with_count` covers the `Vector3`-style array
|
||||
(D-BAST-004).
|
||||
- **Records (count-prefixed string-keyed maps).**
|
||||
`record_of_uint32` covers the `TRecord` shape.
|
||||
- **Untrusted schema input.** `malformed_document_produces_schema_error_not_panic`
|
||||
confirms a malformed BAST document surfaces as `AlkTypeError::Schema`,
|
||||
not a panic (AGENTS.md §3).
|
||||
- **Basic cases** (chunk header, int8/uint32 ranges, string/bytes
|
||||
`maxLength`, timestamp, bool, short buffer) all pass — if a couple of
|
||||
basic examples work, all of them do, since the validator is a flat
|
||||
per-kind dispatch with no per-kind special-casing beyond the range
|
||||
bounds.
|
||||
|
||||
### How the validator works
|
||||
|
||||
The validator is a single recursive function
|
||||
(`validate_typeref`) that dispatches on the BAST `kind`. Each arm
|
||||
checks the value-domain constraint for that kind and, for composites,
|
||||
recurses into the child type definitions. The materializer (also
|
||||
implemented in the POC) guarantees structural correctness — bounds,
|
||||
UTF-8, bool byte, discriminator lookup, all fields present — so the
|
||||
validator only enforces what the materializer cannot:
|
||||
|
||||
| Constraint | Validator arm |
|
||||
|------------|---------------|
|
||||
| Integer range (Int8..Uint64) | `validate_int` / `validate_uint` with `as_i64`/`as_u64` + range check |
|
||||
| Int64/Uint64 (full range) | `validate_int64` / `validate_uint64` (JSON precision caveat per ADR-005) |
|
||||
| Float finiteness (Float32/64) | `validate_float` with `as_f64().is_finite()` |
|
||||
| String `maxLength` (byte length) | `check_string` reads the field-level `maxLength` annotation |
|
||||
| Bytes `maxLength` (array length) | `check_bytes` accepts both `Value::String` and `Value::Array` forms |
|
||||
| RFC 3339 timestamp shape | `validate_timestamp` reuses the same non-strict check as the current engine |
|
||||
| Enum index bounds | `validate_enum` checks `idx < values.len()` — the dead-constraint fix |
|
||||
| Union variant dispatch | `validate_union` reads `__discriminator`, looks up the variant, recurses via `validate_typeref` |
|
||||
| Struct fields | `validate_struct` walks `fields`, requires each declared field present, recurses |
|
||||
| Array count | `validate_array` checks `arr.len() == count` and recurses per element |
|
||||
| Record values | `validate_record` recurses into each value's `values` type |
|
||||
| Boolean | `validate_bool` (materializer already rejects non-0/1 bytes) |
|
||||
|
||||
### Observations for the production implementation
|
||||
|
||||
1. **No `jsonschema` dependency for `validate_bytes`.** The validator
|
||||
only needs `serde_json` (for `Value`) and the BAST document. The
|
||||
`jsonschema` crate is still a direct dependency for `validate_json`
|
||||
and for validating BAST documents against the BAST meta-schema, but
|
||||
the `validate_bytes` path no longer touches it. This is a small wasm
|
||||
binary-size win in addition to the architecture simplification.
|
||||
|
||||
2. **`$ref` resolution is a single hash lookup.** The POC's
|
||||
`resolve_ref_or_inline` handles only `#/$defs/Name` pointers — the
|
||||
only form BAST allows. The current engine's `normalize_refs` /
|
||||
`inline_union_variant_refs` / `resolve_ref_or_inline` machinery for
|
||||
bare-name refs and inlined union variants is no longer needed: BAST
|
||||
`$ref`s are always full JSON Pointers, and union variant refs are
|
||||
resolved lazily by the validator (the materializer already does this
|
||||
for the read path). The `inline_union_variant_refs` compile step can
|
||||
be removed entirely.
|
||||
|
||||
3. **The validator is ~250 lines.** The 19 custom keyword validators
|
||||
(`src/validation.rs`) plus the macro definitions are ~500 lines and
|
||||
require the `jsonschema::Keyword` trait plumbing (factory closures,
|
||||
`Box<dyn Keyword>`, sub-validator construction at factory time). The
|
||||
BAST-native validator is a flat match — no factories, no trait
|
||||
objects, no sub-validator pre-computation. The recursion is direct.
|
||||
|
||||
4. **The `AlkTypeError::Validation` variant still wraps
|
||||
`jsonschema::ValidationError<'static>`.** The POC uses
|
||||
`jsonschema::ValidationError::custom` to construct these so the
|
||||
error type is unchanged. The production version could either keep
|
||||
this (simplest — `ValidationError::custom` is public and
|
||||
`'static`) or introduce a small `Validation(String)` shape to drop
|
||||
the `jsonschema` dependency from the error type. The latter is a
|
||||
public-API change (the `Validation` variant's payload type changes),
|
||||
so it's a semver-relevant decision to make at pivot time, not in the
|
||||
POC.
|
||||
|
||||
5. **The materializer and validator share the BAST-walking code
|
||||
structure.** Both walk the same `kind`/`fields`/`mapping` tree. The
|
||||
production refactor could share a typed BAST tree (a small
|
||||
`BastNode` enum) between them so the walk is parsed once. The POC
|
||||
parses lazily from the raw JSON in both passes to keep the model
|
||||
honest; a typed tree is a straightforward follow-on optimization, not
|
||||
a risk.
|
||||
|
||||
### Verdict
|
||||
|
||||
The "how do we reproduce the same behavior?" question is answered:
|
||||
walk the BAST tree the same way the materializer does, checking the
|
||||
same value-domain constraints the custom keyword validators check
|
||||
today. The model is a strict simplification — fewer moving parts, no
|
||||
`jsonschema` integration on the bytes path, no compile-time
|
||||
`inline_union_variant_refs` step, no factory closures or trait
|
||||
objects, and the enum dead-constraint is fixed as a side effect.
|
||||
|
||||
The POC does not wire into `AlkTypeEngine::validate_bytes` — that is
|
||||
the production refactor (Phase 1 step 5 in the Migration Path), which
|
||||
replaces `validation::build_validator` usage on the bytes path with
|
||||
the BAST-native validator. The POC's job was to de-risk the model
|
||||
before that refactor; that job is done.
|
||||
+1454
File diff suppressed because it is too large.
Load diff
@@ -28,6 +28,7 @@
|
||||
|
||||
#[macro_use]
|
||||
mod macros;
|
||||
pub mod bast_poc;
|
||||
pub mod builder;
|
||||
pub mod data_access;
|
||||
pub mod engine;
|
||||
|
||||
Reference in new issue
Block a user