Add BAST-native validator POC for validate_bytes

Implements the D-BAST-006 validation model as a self-contained module
(`src/bast_poc.rs`) that does not touch the production schema /
materializer / validator paths. Reuses only `data_access`,
`AlkTypeError`, and `Endian` — the BAST parser, packed-mode
materializer, and native validator are all from scratch, which is the
point: prove the model works end-to-end before refactoring.

The POC confirms the hypothesis from the pivot doc: a recursive walker
over the BAST type tree fully replaces the 19 custom keyword
validators on the `validate_bytes` path, recovers the OQ-008 union
variant dispatch, and fixes the enum-membership dead constraint on the
bytes path — all without `jsonschema` custom keywords and without an
external JSON Schema.

Coverage (20 tests, all passing):
- basic: chunk header, int8/uint32 ranges, string/bytes maxLength,
  timestamp, bool, short buffer
- hard: union byte-offset disc + maxLength inside variant (OQ-008),
  union field-name disc + maxLength inside variant, enum index-bounds
  fix, nested struct→union→struct, array of structs with count,
  record, untrusted-schema error-not-panic

Verification:
- cargo test --release (416 tests: 330 lib + 86 integration)
- cargo clippy --all-targets -- -D warnings
- cargo build --target wasm32-unknown-unknown --release
- cargo doc --no-deps

Findings and verdict recorded in docs/research/bast-pivot.md
§"POC Result — BAST-native validator".
This commit is contained in:
glm-5.2 committed 2026-08-15 09:29:14 +00:00
1 parent 19f8162f1a
commit f371fe4e94
3 files changed
+1605

No files matched your search

+150
View File
@@ -1165,3 +1165,153 @@ readers/writers from BAST documents for these crates.
- `/workspace/@alkimiadev/typebox-rs/src/codegen/` — Rust/TypeScript codegen from schemas (reference architecture)
- `/workspace/alknet-typedef-poc/tests/sftp_roundtrip_test.rs` — SFTP POC proving byte-identical output (to be replicated with BAST)
- `/workspace/@alkdev/alkcall/src/channels/wire.rs` — hand-rolled chunk header (target for BAST replacement)
## POC Result — BAST-native validator
**Status: succeeded.** The POC is on branch `bast-validator-poc` in
`src/bast_poc.rs` (20 tests, all passing; full crate suite — 416 tests —
green; `cargo clippy --all-targets -- -D warnings` clean;
`cargo build --target wasm32-unknown-unknown --release` clean).
The POC implements the `validate_bytes` validation model from
[D-BAST-006](#d-bast-006-validate_bytes-validation-model) as a
self-contained module that does **not** touch the production schema /
materializer / validator paths. It reuses only `data_access` (read
primitives), `AlkTypeError` (error type), and `Endian`. The BAST
document parser, a packed-mode materializer, and the BAST-native
validator are all implemented from scratch — that is the point: prove
the model works end-to-end before refactoring the production code.
### What the POC proves
The hypothesis from the [POC section](#poc-bast-native-validator-for-validate_bytes)
is confirmed: a recursive walker over the BAST type tree fully
replaces the 19 custom keyword validators on the `validate_bytes` path,
including the OQ-008 union variant dispatch, and fixes the
enum-membership dead constraint — all without `jsonschema` custom
keywords and without an external JSON Schema.
The hard cases that were the actual de-risking targets all pass:
- **Union byte-offset discriminator + `maxLength` inside a variant
(OQ-008).** `union_byte_disc_max_length_inside_variant_enforced`
materializes a union with two `$ref` variants, dispatches on a
byte-offset `uint8` discriminator, and enforces `maxLength` on a
`bytes` field inside the selected variant. The validator reads
`__discriminator`, looks up the variant's BAST definition, and
recurses — same behavior as the current `UnionValidator`'s
per-variant sub-validators, but with no `jsonschema` involvement.
- **Union field-name discriminator + `maxLength` inside a variant.**
`union_field_disc_max_length_inside_variant_enforced` covers the
typedef.ts-style discriminator (a length-prefixed string field
selects the variant). Same recursion model.
- **Enum index-bounds fix.** `enum_index_out_of_bounds_rejected`
exercises the constraint that is **broken in the current engine**
(the built-in `enum` keyword checks string membership; the
materializer emits `Value::Number(index)`, which never matches — a
dead constraint). The BAST-native validator checks the materialized
index against the `values` array bounds (0..len-1), which is the
correct validation for a binary enum encoded as an index. Net
improvement, not a regression.
- **Nested struct wrapping a union wrapping a struct.**
`nested_struct_with_union_variant` confirms the recursion composes
through multiple type layers.
- **Arrays of fixed-size structs with `count`.**
`array_of_structs_with_count` covers the `Vector3`-style array
(D-BAST-004).
- **Records (count-prefixed string-keyed maps).**
`record_of_uint32` covers the `TRecord` shape.
- **Untrusted schema input.** `malformed_document_produces_schema_error_not_panic`
confirms a malformed BAST document surfaces as `AlkTypeError::Schema`,
not a panic (AGENTS.md §3).
- **Basic cases** (chunk header, int8/uint32 ranges, string/bytes
`maxLength`, timestamp, bool, short buffer) all pass — if a couple of
basic examples work, all of them do, since the validator is a flat
per-kind dispatch with no per-kind special-casing beyond the range
bounds.
### How the validator works
The validator is a single recursive function
(`validate_typeref`) that dispatches on the BAST `kind`. Each arm
checks the value-domain constraint for that kind and, for composites,
recurses into the child type definitions. The materializer (also
implemented in the POC) guarantees structural correctness — bounds,
UTF-8, bool byte, discriminator lookup, all fields present — so the
validator only enforces what the materializer cannot:
| Constraint | Validator arm |
|------------|---------------|
| Integer range (Int8..Uint64) | `validate_int` / `validate_uint` with `as_i64`/`as_u64` + range check |
| Int64/Uint64 (full range) | `validate_int64` / `validate_uint64` (JSON precision caveat per ADR-005) |
| Float finiteness (Float32/64) | `validate_float` with `as_f64().is_finite()` |
| String `maxLength` (byte length) | `check_string` reads the field-level `maxLength` annotation |
| Bytes `maxLength` (array length) | `check_bytes` accepts both `Value::String` and `Value::Array` forms |
| RFC 3339 timestamp shape | `validate_timestamp` reuses the same non-strict check as the current engine |
| Enum index bounds | `validate_enum` checks `idx < values.len()` — the dead-constraint fix |
| Union variant dispatch | `validate_union` reads `__discriminator`, looks up the variant, recurses via `validate_typeref` |
| Struct fields | `validate_struct` walks `fields`, requires each declared field present, recurses |
| Array count | `validate_array` checks `arr.len() == count` and recurses per element |
| Record values | `validate_record` recurses into each value's `values` type |
| Boolean | `validate_bool` (materializer already rejects non-0/1 bytes) |
### Observations for the production implementation
1. **No `jsonschema` dependency for `validate_bytes`.** The validator
only needs `serde_json` (for `Value`) and the BAST document. The
`jsonschema` crate is still a direct dependency for `validate_json`
and for validating BAST documents against the BAST meta-schema, but
the `validate_bytes` path no longer touches it. This is a small wasm
binary-size win in addition to the architecture simplification.
2. **`$ref` resolution is a single hash lookup.** The POC's
`resolve_ref_or_inline` handles only `#/$defs/Name` pointers — the
only form BAST allows. The current engine's `normalize_refs` /
`inline_union_variant_refs` / `resolve_ref_or_inline` machinery for
bare-name refs and inlined union variants is no longer needed: BAST
`$ref`s are always full JSON Pointers, and union variant refs are
resolved lazily by the validator (the materializer already does this
for the read path). The `inline_union_variant_refs` compile step can
be removed entirely.
3. **The validator is ~250 lines.** The 19 custom keyword validators
(`src/validation.rs`) plus the macro definitions are ~500 lines and
require the `jsonschema::Keyword` trait plumbing (factory closures,
`Box<dyn Keyword>`, sub-validator construction at factory time). The
BAST-native validator is a flat match — no factories, no trait
objects, no sub-validator pre-computation. The recursion is direct.
4. **The `AlkTypeError::Validation` variant still wraps
`jsonschema::ValidationError<'static>`.** The POC uses
`jsonschema::ValidationError::custom` to construct these so the
error type is unchanged. The production version could either keep
this (simplest — `ValidationError::custom` is public and
`'static`) or introduce a small `Validation(String)` shape to drop
the `jsonschema` dependency from the error type. The latter is a
public-API change (the `Validation` variant's payload type changes),
so it's a semver-relevant decision to make at pivot time, not in the
POC.
5. **The materializer and validator share the BAST-walking code
structure.** Both walk the same `kind`/`fields`/`mapping` tree. The
production refactor could share a typed BAST tree (a small
`BastNode` enum) between them so the walk is parsed once. The POC
parses lazily from the raw JSON in both passes to keep the model
honest; a typed tree is a straightforward follow-on optimization, not
a risk.
### Verdict
The "how do we reproduce the same behavior?" question is answered:
walk the BAST tree the same way the materializer does, checking the
same value-domain constraints the custom keyword validators check
today. The model is a strict simplification — fewer moving parts, no
`jsonschema` integration on the bytes path, no compile-time
`inline_union_variant_refs` step, no factory closures or trait
objects, and the enum dead-constraint is fixed as a side effect.
The POC does not wire into `AlkTypeEngine::validate_bytes` — that is
the production refactor (Phase 1 step 5 in the Migration Path), which
replaces `validation::build_validator` usage on the bytes path with
the BAST-native validator. The POC's job was to de-risk the model
before that refactor; that job is done.
+1454
View File
File diff suppressed because it is too large. Load diff
+1
View File
@@ -28,6 +28,7 @@
#[macro_use]
mod macros;
pub mod bast_poc;
pub mod builder;
pub mod data_access;
pub mod engine;