Converts the open question into a closed decision. Rationale: consumer
ergonomics on the combined validate_json + validate_bytes path — one
uniform payload type means one match arm downstream. Option 2
(Validation(String)) would force validate_json to flatten its
structured errors to a String, losing information on the richer path to
accommodate the less rich one. The no_std/minimal-build angle that
option 2 was meant to enable is moot: validate_json requires jsonschema
regardless, so a bytes-only no_std build already has to give up
validate_json as a separate larger decision; dropping the type from one
error variant doesn't unlock it.
Updates Phase 1 step 5 to reference D-BAST-009 for the error
construction pattern, and rewrites POC Result observation 4 from
'deferred decision' to 'decided — see D-BAST-009'.
No semver-relevant change to the Validation variant. Doc-only.
Adds the POC Result section (POC on branch bast-validator-poc, commit
f371fe4 — 20/20 tests, full 416-test suite green, clippy/wasm/doc clean).
Hypothesis confirmed: a recursive walker over the BAST type tree fully
replaces the 19 custom keyword validators on the validate_bytes path,
recovers OQ-008 union variant dispatch, and fixes the enum-membership
dead constraint. The POC code is reference scaffolding on the branch
and is not merged to main — it is superseded by Phase 1 step 5.
Marks Phase 1 step 2 and the POC Scope section as done with pointers to
the result section.
Elevates the deferred AlkTypeError::Validation payload-shape question
to OQ-BAST-001: keep jsonschema::ValidationError<'static> (POC choice,
simplest, dependency stays) vs introduce Validation(String) (drops
jsonschema from the error type; semver-relevant public-API change).
Decision belongs to the production refactor.
Verification: doc-only change, no code touched.
- Rewrite Validator Split around BAST-native validator for
validate_bytes (walks BAST, checks value-domain constraints, no
external JSON Schema needed); validate_json uses standard
jsonschema::Validator from consumer-provided JSON Schema
- Add targeted POC: BAST-native validator replacing 19 custom keyword
validators on the bytes path, verified via existing test suite
- Fix meta-schema: require count on arrays (variable-element arrays
deferred per OQ-001), add optional fields array to UnionDef for
field-name discriminators
- Remove lying no-count array example, replace with deferred note
- Document dead enum constraint (materialized index never matches
string-membered enum); BAST-native validator fixes it via index
bounds check
- Resolve all 7 OQs + 3 spec gaps as D-BAST-001 through D-BAST-008
- Update Migration Path, Risks table, Engine internals to reflect
BAST-native validator
- Clarify 'no pocs needed' was an overcorrection: layout swap needs
no POC, but the validation model does
Verification: docs-only change, no code affected
- Remove the four proposed POCs: they were implementation smoke tests,
not de-risking probes. The pivot is a backend swap on a proven layout
engine; byte-identity is already proven and the layout code is
unchanged, so there is nothing empirical left to de-risk.
- Remove the recursive TreeNode example and the recursion mention in
design principle 2: recursion is not a binary-layout concern and the
engine has no cycle detection.
- Add a Spec Gaps section: validate_bytes semantics after keyword
validator removal (UnionValidator variant dispatch regression),
arrays of variable-length elements (engine rejects them), and
field-name discriminator unions (meta-schema cannot express them).
- Reframe the engine change as an accessor-layer refactor: the
walkers' (kind, field list, annotations) reads change; everything
beneath them carries over unchanged.
Correct the validator split section to clarify that jsonschema is
not removed — it remains the JSON Schema validator for both paths
(BAST meta-schema validation and standard JSON payload validation).
Only the custom keyword registration path is removed. Also fix the
build_validator and validate_json entries in the engine changes
table to reflect that they are repurposed, not removed.
Proposes replacing custom JSON Schema keywords with a standalone
kind-based JSON format (BAST) using / for composition.
Covers format design, meta-schema, engine changes, validator split,
codegen future, ABI adapter potential, migration path, 7 open
questions, and 4 proposed POCs.
POC: /workspace/alktype-builder-poc/ (18/18 tests pass, findings in
docs/research/alktype-builder-poc/findings.md). Round 2 adds the SFTP
Packet validate_bytes tests (7 new: valid Init/Read/Write/Status,
short buffer, unknown discriminator, over-maxLength Bytes).
Open questions resolved (OQ-004 through OQ-008):
- OQ-004: Discriminator::Field name is String (already implemented;
docs updated to mark resolved)
- OQ-005: Both union discriminator kinds return the same shape:
{__discriminator, ...variant-fields}. Field-name path also had a
real offset bug (returned start, not end) - fixed.
- OQ-006: builder.md Example 3 now wraps the Union in a
Schema::struct_().field("payload", ...) and merges $defs via
Definitions::merge_into (matches the engine's AlkType:Struct-at-root
constraint and the SFTP wire shape)
- OQ-007: Bytes materialization is array-of-u8 (Value::Array of
Value::Number, one entry per byte 0..=255). BytesValidator accepts
both Value::String (validate_json) and Value::Array (validate_bytes).
maxLength = max byte count. Replaces the lossy from_utf8_lossy path
that corrupted non-UTF-8 bytes and broke maxLength semantics.
- OQ-008 (new): UnionValidator now dispatches to variant schemas via
sub-validators built at factory time. AlkTypeEngine::compile calls
schema::inline_union_variant_refs before build_validator to inline
$refs in union mapping entries (necessary because union_factory
receives the union node, but $defs live at the schema root).
Production-readiness fixes in src/ (no stubs/hedges in a published crate):
- materialize.rs: Record stub -> full count-prefixed key/value pair
implementation per schema-layer.md TRecord
- materialize.rs: root_of() was broken (returned the current node, not
the schema root) -> root schema threaded through every recursive call
so resolve_ref_or_inline can resolve $refs for nested composites
- builder.rs: LengthPrefixed encoding setter was a no-op when the
keyword was already in object form -> complete the branch (updates
the encoding entry in place for both LengthPrefixed and OffsetIndirect)
- builder.rs, engine.rs: POC-referencing comments cleaned up; the
round-trip test's or_else fallback (papering over write_field being
aligned-only) replaced with direct byte writes
Documentation:
- builder.md: Example 3 updated; Discriminator::Field spec shows String;
Open Questions section updated (OQ-004 resolved)
- validation.md: AlkType:Bytes and AlkType:Union validator descriptions
updated for array-of-u8 form and variant dispatch
- open-questions.md: OQ-004/005/006/007/008 marked resolved; new
Validation theme entries
- questions/004-008: individual OQ files updated with resolutions
- findings.md: round 2 results documented
Verification:
- cargo test: 369 -> 391 tests pass (22 new: 14 materialize, 5
inline_union_variant_refs, 3 validation/builder)
- cargo clippy --all-targets: clean
- POC: 11 -> 18 tests (7 new SFTP Packet tests); all pass
Four open questions were scattered inline in builder.md and the POC
findings doc. Moved them into the central OQ tracker under
docs/architecture/questions/ and updated the index:
- OQ-004: Discriminator::Field name type (&str vs String) — raised in
builder.md during ADR-009 spec drafting
- OQ-005: Union materialization shape (byte-offset vs field-name
consistency) — raised in the POC findings
- OQ-006: Builder spec Example 3 wrap Union in Struct — raised in the
POC findings (doc fix; engine requires AlkType:Struct at top level)
- OQ-007: Bytes materialization lossy UTF-8 — raised in the POC
findings (blocks SFTP use case for validate_bytes)
Index updates:
- open-questions.md: new 'Schema Construction' and 'Validation' theme
groups; new 'Open' section for active investigation targets (distinct
from 'Deferred / Blocked' which holds scope-parked OQs)
- README.md: OQ table extended with OQ-004 through OQ-007
- builder.md: inline OQ-004 replaced with a tracker reference
- findings.md: inline OQ-005/006/007 replaced with tracker references
Doc-only change; 369 tests pass.
POC: /workspace/alktype-builder-poc/ (11/11 tests pass, findings in
docs/research/alktype-builder-poc/findings.md). The POC code lives outside
the repo per the internal dev convention.
Implementation:
- src/builder.rs: Schema, Definitions, Discriminator types. Constructors
for all 19 AlkType kinds + standard JSON Schema types (object/array/
string/integer/number/boolean/null/any). Setters for ADR-003 annotations
(endian/align/encoding/max_length), composite builders (field/required/
items/mapping), and standard JSON Schema constraints (minimum/maximum/
minLength/minItems/maxItems/format/title/description). 16 unit tests.
- src/materialize.rs: materialize_packed and materialize_aligned functions
that walk a schema + buffer to produce a serde_json::Value tree. Recurses
into Struct, Array, Union (byte-offset discriminator). Record is stubbed
(deferred for the POC scope).
- src/engine.rs: AlkTypeEngine::validate_bytes(&[u8]) added (ADR-010).
Dispatches on layout mode, materializes Value, then validates against
the existing jsonschema validator. 7 unit tests.
- src/lib.rs: pub mod builder, pub mod materialize; re-exports Schema,
Definitions, Discriminator.
Verification:
- cargo test: 346 -> 369 tests pass (23 new: 16 builder, 7 validate_bytes)
- cargo clippy --all-targets -- -D warnings: clean
- POC (11 tests): builder round-trip + validate_bytes (packed + aligned) +
validate_json for call payloads; all pass
Findings:
- Top-level schema must be AlkType:Struct (existing constraint); unions are
field types within a struct. Builder spec Example 3 needs a doc fix.
- Builder field order preserved (preserve_order feature, load-bearing for
packed mode).
- validate_bytes correctly distinguishes Access (read phase) from
Validation (validate phase) errors, with field paths.
- validate_json path unchanged for call payloads.
Open questions surfaced (OQ-005, OQ-006, OQ-007) tracked in findings.md;
to resolve before the SFTP Packet POC round.
glm-5.2
deepseek-v4-pro