Resolve v0.1.0 open questions and fix production-readiness issues

POC: /workspace/alktype-builder-poc/ (18/18 tests pass, findings in
docs/research/alktype-builder-poc/findings.md). Round 2 adds the SFTP
Packet validate_bytes tests (7 new: valid Init/Read/Write/Status,
short buffer, unknown discriminator, over-maxLength Bytes).

Open questions resolved (OQ-004 through OQ-008):
- OQ-004: Discriminator::Field name is String (already implemented;
  docs updated to mark resolved)
- OQ-005: Both union discriminator kinds return the same shape:
  {__discriminator, ...variant-fields}. Field-name path also had a
  real offset bug (returned start, not end) - fixed.
- OQ-006: builder.md Example 3 now wraps the Union in a
  Schema::struct_().field("payload", ...) and merges $defs via
  Definitions::merge_into (matches the engine's AlkType:Struct-at-root
  constraint and the SFTP wire shape)
- OQ-007: Bytes materialization is array-of-u8 (Value::Array of
  Value::Number, one entry per byte 0..=255). BytesValidator accepts
  both Value::String (validate_json) and Value::Array (validate_bytes).
  maxLength = max byte count. Replaces the lossy from_utf8_lossy path
  that corrupted non-UTF-8 bytes and broke maxLength semantics.
- OQ-008 (new): UnionValidator now dispatches to variant schemas via
  sub-validators built at factory time. AlkTypeEngine::compile calls
  schema::inline_union_variant_refs before build_validator to inline
  $refs in union mapping entries (necessary because union_factory
  receives the union node, but $defs live at the schema root).

Production-readiness fixes in src/ (no stubs/hedges in a published crate):
- materialize.rs: Record stub -> full count-prefixed key/value pair
  implementation per schema-layer.md TRecord
- materialize.rs: root_of() was broken (returned the current node, not
  the schema root) -> root schema threaded through every recursive call
  so resolve_ref_or_inline can resolve $refs for nested composites
- builder.rs: LengthPrefixed encoding setter was a no-op when the
  keyword was already in object form -> complete the branch (updates
  the encoding entry in place for both LengthPrefixed and OffsetIndirect)
- builder.rs, engine.rs: POC-referencing comments cleaned up; the
  round-trip test's or_else fallback (papering over write_field being
  aligned-only) replaced with direct byte writes

Documentation:
- builder.md: Example 3 updated; Discriminator::Field spec shows String;
  Open Questions section updated (OQ-004 resolved)
- validation.md: AlkType:Bytes and AlkType:Union validator descriptions
  updated for array-of-u8 form and variant dispatch
- open-questions.md: OQ-004/005/006/007/008 marked resolved; new
  Validation theme entries
- questions/004-008: individual OQ files updated with resolutions
- findings.md: round 2 results documented

Verification:
- cargo test: 369 -> 391 tests pass (22 new: 14 materialize, 5
  inline_union_variant_refs, 3 validation/builder)
- cargo clippy --all-targets: clean
- POC: 11 -> 18 tests (7 new SFTP Packet tests); all pass
This commit is contained in:
glm-5.2 committed 2026-08-11 07:28:24 +00:00
1 parent c6893eece8
commit c0217d91a8
15 files changed
+1553 -493

No files matched your search

+167
View File
@@ -467,6 +467,64 @@ fn normalize_refs_recursive(node: &mut Value) {
}
}
/// Inline `$ref`s in `AlkType:Union` `mapping` entries by resolving them
/// against the schema root and replacing each `$ref`-bearing variant with
/// the resolved schema (in place). This runs after [`normalize_refs`] and
/// before [`crate::validation::build_validator`].
///
/// The `UnionValidator`'s factory builds a sub-validator for each variant
/// at validator-construction time. The factory receives the union node as
/// `parent`, but `$defs` live at the schema root — not on the union node.
/// Without inlining, the factory can't resolve `$ref`s like
/// `"#/$defs/Init"` because the union node doesn't contain `$defs`.
/// Inlining the refs before validator construction sidesteps this: each
/// variant in the `mapping` becomes a full inline schema, so the factory
/// can build a sub-validator directly.
///
/// Only `AlkType:Union` mappings are inlined. Other `$ref`s (e.g. in
/// `properties` or `items`) are left in place — the materializer resolves
/// them at read time via [`resolve_ref_or_inline`] against the schema
/// root, and the jsonschema built-in validator resolves them via its own
/// `$ref` resolution (which has access to the full schema root).
///
/// Idempotent: a mapping whose variants are already inline (no `$ref`)
/// is left unchanged.
pub fn inline_union_variant_refs(root: &mut Value) {
// Collect (path-to-union-mapping, ref-path, variant-key) tuples by
// walking the tree with immutable access, then resolve and apply
// the inlining with mutable access. This avoids the borrow conflict
// of holding both &root and &mut root simultaneously.
let root_clone = root.clone();
inline_union_variant_refs_recursive(root, &root_clone);
}
fn inline_union_variant_refs_recursive(node: &mut Value, root: &Value) {
if let Value::Object(obj) = node {
// Check if this node is an AlkType:Union with a mapping.
let is_union = obj
.keys()
.any(|k| k == "AlkType:Union");
if is_union {
if let Some(Value::Object(mapping)) = obj.get_mut("mapping") {
for (_key, variant) in mapping.iter_mut() {
if let Some(Value::String(ref_path)) = variant.get("$ref").cloned() {
if let Some(resolved) = resolve_ref(root, &ref_path) {
*variant = resolved.clone();
}
}
}
}
}
for value in obj.values_mut() {
inline_union_variant_refs_recursive(value, root);
}
} else if let Value::Array(arr) = node {
for item in arr.iter_mut() {
inline_union_variant_refs_recursive(item, root);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -810,6 +868,115 @@ mod tests {
);
}
// ----- inline_union_variant_refs tests -----
#[test]
fn inline_union_variant_refs_inlines_mapping_refs() {
let mut schema = json!({
"AlkType:Struct": true,
"properties": {
"payload": {
"AlkType:Union": true,
"discriminator": { "kind": "byte", "offset": 0, "type": "AlkType:Uint8" },
"mapping": {
"5": {"$ref": "#/$defs/Read"},
"6": {"$ref": "#/$defs/Write"}
}
}
},
"$defs": {
"Read": { "AlkType:Struct": true, "properties": { "id": { "AlkType:Uint32": true } } },
"Write": { "AlkType:Struct": true, "properties": { "n": { "AlkType:Uint16": true } } }
}
});
inline_union_variant_refs(&mut schema);
// The mapping entries should now be the full inline schemas.
let mapping = &schema["properties"]["payload"]["mapping"];
assert_eq!(
mapping["5"],
json!({ "AlkType:Struct": true, "properties": { "id": { "AlkType:Uint32": true } } })
);
assert_eq!(
mapping["6"],
json!({ "AlkType:Struct": true, "properties": { "n": { "AlkType:Uint16": true } } })
);
// $defs are left in place (the materializer may still use them).
assert!(schema["$defs"].is_object());
}
#[test]
fn inline_union_variant_refs_leaves_inline_variants_unchanged() {
let mut schema = json!({
"AlkType:Union": true,
"discriminator": { "kind": "byte", "offset": 0, "type": "AlkType:Uint8" },
"mapping": {
"5": { "AlkType:Struct": true, "properties": { "id": { "AlkType:Uint32": true } } }
}
});
let before = schema.clone();
inline_union_variant_refs(&mut schema);
assert_eq!(schema, before, "inline variants should be unchanged");
}
#[test]
fn inline_union_variant_refs_is_idempotent() {
let mut schema = json!({
"AlkType:Union": true,
"discriminator": { "kind": "byte", "offset": 0, "type": "AlkType:Uint8" },
"mapping": { "5": {"$ref": "#/$defs/Read"} },
"$defs": { "Read": { "AlkType:Struct": true, "properties": { "id": { "AlkType:Uint32": true } } } }
});
inline_union_variant_refs(&mut schema);
let after_first = schema.clone();
inline_union_variant_refs(&mut schema);
assert_eq!(schema, after_first, "second inlining should be a no-op");
}
#[test]
fn inline_union_variant_refs_handles_nested_union() {
// A struct containing a union whose variants are $refs. The
// inlining walks into the struct's properties and inlines the
// union's mapping refs.
let mut schema = json!({
"AlkType:Struct": true,
"properties": {
"outer": {
"AlkType:Struct": true,
"properties": {
"inner_union": {
"AlkType:Union": true,
"discriminator": { "kind": "byte", "offset": 0, "type": "AlkType:Uint8" },
"mapping": { "1": {"$ref": "#/$defs/Init"} }
}
}
}
},
"$defs": { "Init": { "AlkType:Struct": true, "properties": { "v": { "AlkType:Uint32": true } } } }
});
inline_union_variant_refs(&mut schema);
let mapping = &schema["properties"]["outer"]["properties"]["inner_union"]["mapping"];
assert_eq!(
mapping["1"],
json!({ "AlkType:Struct": true, "properties": { "v": { "AlkType:Uint32": true } } })
);
}
#[test]
fn inline_union_variant_refs_leaves_non_union_refs_in_place() {
// $refs in `properties` (not in a union mapping) are left in
// place — the materializer resolves them at read time.
let mut schema = json!({
"AlkType:Struct": true,
"properties": {
"field": {"$ref": "#/$defs/SomeType"}
},
"$defs": { "SomeType": { "AlkType:Uint32": true } }
});
let before = schema.clone();
inline_union_variant_refs(&mut schema);
assert_eq!(schema, before, "non-union refs should be unchanged");
}
#[test]
fn as_str_round_trips_for_all_kinds() {
for kind in [