Implement builder (ADR-009) and validate_bytes (ADR-010) for v0.1.0

POC: /workspace/alktype-builder-poc/ (11/11 tests pass, findings in
docs/research/alktype-builder-poc/findings.md). The POC code lives outside
the repo per the internal dev convention.

Implementation:
- src/builder.rs: Schema, Definitions, Discriminator types. Constructors
  for all 19 AlkType kinds + standard JSON Schema types (object/array/
  string/integer/number/boolean/null/any). Setters for ADR-003 annotations
  (endian/align/encoding/max_length), composite builders (field/required/
  items/mapping), and standard JSON Schema constraints (minimum/maximum/
  minLength/minItems/maxItems/format/title/description). 16 unit tests.
- src/materialize.rs: materialize_packed and materialize_aligned functions
  that walk a schema + buffer to produce a serde_json::Value tree. Recurses
  into Struct, Array, Union (byte-offset discriminator). Record is stubbed
  (deferred for the POC scope).
- src/engine.rs: AlkTypeEngine::validate_bytes(&[u8]) added (ADR-010).
  Dispatches on layout mode, materializes Value, then validates against
  the existing jsonschema validator. 7 unit tests.
- src/lib.rs: pub mod builder, pub mod materialize; re-exports Schema,
  Definitions, Discriminator.

Verification:
- cargo test: 346 -> 369 tests pass (23 new: 16 builder, 7 validate_bytes)
- cargo clippy --all-targets -- -D warnings: clean
- POC (11 tests): builder round-trip + validate_bytes (packed + aligned) +
  validate_json for call payloads; all pass

Findings:
- Top-level schema must be AlkType:Struct (existing constraint); unions are
  field types within a struct. Builder spec Example 3 needs a doc fix.
- Builder field order preserved (preserve_order feature, load-bearing for
  packed mode).
- validate_bytes correctly distinguishes Access (read phase) from
  Validation (validate phase) errors, with field paths.
- validate_json path unchanged for call payloads.

Open questions surfaced (OQ-005, OQ-006, OQ-007) tracked in findings.md;
to resolve before the SFTP Packet POC round.
This commit is contained in:
glm-5.2 committed 2026-08-11 05:49:09 +00:00
1 parent 1a8a44ed0e
commit 5588278451
5 files changed
+1542

No files matched your search

+672
View File
@@ -0,0 +1,672 @@
//! Fluent Rust builder for constructing alktype JSON Schemas at runtime.
//!
//! Produces `serde_json::Value` — the same form [`crate::AlkTypeEngine`]
//! consumes. Covers the 19 `AlkType:*` kinds and the standard JSON Schema
//! keywords needed for operation payload schemas. Decided in ADR-009;
//! spec in `docs/architecture/builder.md`.
//!
//! The builder is a thin construction layer over the JSON form. There is
//! no typed `Schema` enum; `.build()` returns a `Value`. Setters return
//! `Self` for chaining. Field order is load-bearing for binary layouts
//! (`serde_json`'s `preserve_order` feature preserves insertion order).
use crate::schema::{AlkTypeKind, Endian, VariableEncoding};
use serde_json::{Map, Value};
const ALKTYPE_PREFIX: &str = "AlkType:";
/// A schema under construction. One constructor per AlkType kind or
/// standard JSON Schema type; setters for annotations and constraints;
/// `.build()` produces the final [`Value`].
///
/// See `docs/architecture/builder.md` for the full API spec.
#[derive(Debug, Clone)]
pub struct Schema {
/// The accumulated JSON object. Built incrementally via setters.
/// Keys preserve insertion order under `serde_json`'s `preserve_order`
/// feature, which is load-bearing for binary layouts (field order
/// = byte order in packed mode).
object: Map<String, Value>,
}
impl Schema {
// -----------------------------------------------------------------
// AlkType kind constructors
// -----------------------------------------------------------------
/// `AlkType:Int8`.
pub fn int8() -> Self {
Self::with_kind(AlkTypeKind::Int8)
}
/// `AlkType:Int16`.
pub fn int16() -> Self {
Self::with_kind(AlkTypeKind::Int16)
}
/// `AlkType:Int32`.
pub fn int32() -> Self {
Self::with_kind(AlkTypeKind::Int32)
}
/// `AlkType:Int64`.
pub fn int64() -> Self {
Self::with_kind(AlkTypeKind::Int64)
}
/// `AlkType:Uint8`.
pub fn uint8() -> Self {
Self::with_kind(AlkTypeKind::Uint8)
}
/// `AlkType:Uint16`.
pub fn uint16() -> Self {
Self::with_kind(AlkTypeKind::Uint16)
}
/// `AlkType:Uint32`.
pub fn uint32() -> Self {
Self::with_kind(AlkTypeKind::Uint32)
}
/// `AlkType:Uint64`.
pub fn uint64() -> Self {
Self::with_kind(AlkTypeKind::Uint64)
}
/// `AlkType:Float32`.
pub fn float32() -> Self {
Self::with_kind(AlkTypeKind::Float32)
}
/// `AlkType:Float64`.
pub fn float64() -> Self {
Self::with_kind(AlkTypeKind::Float64)
}
/// `AlkType:Boolean`.
pub fn boolean() -> Self {
Self::with_kind(AlkTypeKind::Boolean)
}
/// `AlkType:String` (length-prefixed UTF-8).
pub fn string() -> Self {
Self::with_kind(AlkTypeKind::String)
}
/// `AlkType:Bytes` (length-prefixed raw bytes).
pub fn bytes() -> Self {
Self::with_kind(AlkTypeKind::Bytes)
}
/// `AlkType:Timestamp` (length-prefixed RFC 3339 string).
pub fn timestamp() -> Self {
Self::with_kind(AlkTypeKind::Timestamp)
}
/// `AlkType:Struct` (record of fields; add via `.field()`).
pub fn struct_() -> Self {
Self::with_kind(AlkTypeKind::Struct)
}
/// `AlkType:Array` (repeated element).
pub fn array_of(element: Schema) -> Self {
let mut s = Self::with_kind(AlkTypeKind::Array);
s.object.insert("items".to_string(), element.build());
s
}
/// `AlkType:Record` (string-keyed map of values).
pub fn record_of(value: Schema) -> Self {
let mut s = Self::with_kind(AlkTypeKind::Record);
s.object.insert("values".to_string(), value.build());
s
}
/// `AlkType:Union` (tagged union; add variants via `.mapping()`).
pub fn union_(disc: Discriminator) -> Self {
let mut s = Self::with_kind(AlkTypeKind::Union);
match disc {
Discriminator::Byte { offset, disc_type } => {
s.object.insert(
"discriminator".to_string(),
Value::Object(Map::from_iter([
("kind".to_string(), Value::String("byte".to_string())),
("offset".to_string(), Value::from(offset as u64)),
("type".to_string(), Value::String(disc_type.as_str().to_string())),
])),
);
}
Discriminator::Field { name } => {
s.object.insert(
"discriminator".to_string(),
Value::Object(Map::from_iter([
("kind".to_string(), Value::String("field".to_string())),
("name".to_string(), Value::String(name)),
])),
);
}
}
s
}
/// `AlkType:Enum` with the provided values in declaration order.
/// The binary representation is a `u32` index into this list.
pub fn enum_of(values: &[&str]) -> Self {
let mut s = Self::with_kind(AlkTypeKind::Enum);
s.object.insert(
"enum".to_string(),
Value::Array(values.iter().map(|v| Value::String((*v).to_string())).collect()),
);
s
}
fn with_kind(kind: AlkTypeKind) -> Self {
let mut object = Map::new();
object.insert(kind.as_str().to_string(), Value::Bool(true));
Self { object }
}
// -----------------------------------------------------------------
// Standard JSON Schema type constructors (no AlkType:* kinds)
// -----------------------------------------------------------------
/// Standard `type: "object"`. Fields via `.field()`.
pub fn object() -> Self {
Self::with_type("object")
}
/// Standard `type: "array"`. Items via `.items()`.
pub fn array() -> Self {
Self::with_type("array")
}
/// Standard `type: "string"`.
pub fn string_() -> Self {
Self::with_type("string")
}
/// Standard `type: "integer"`.
pub fn integer() -> Self {
Self::with_type("integer")
}
/// Standard `type: "number"`.
pub fn number() -> Self {
Self::with_type("number")
}
/// Standard `type: "boolean"`.
pub fn boolean_() -> Self {
Self::with_type("boolean")
}
/// Standard `type: "null"`.
pub fn null() -> Self {
Self::with_type("null")
}
/// No type constraint (the `{}` form).
pub fn any() -> Self {
Self { object: Map::new() }
}
fn with_type(type_str: &str) -> Self {
let mut object = Map::new();
object.insert("type".to_string(), Value::String(type_str.to_string()));
Self { object }
}
/// Adopt an existing JSON Schema `Value` as a `Schema` for composition.
pub fn from_value(value: Value) -> Self {
Self {
object: value
.as_object()
.cloned()
.unwrap_or_else(|| {
let mut m = Map::new();
m.insert("<root>".to_string(), value);
m
}),
}
}
/// Produce a `{"$ref": "#/$defs/<name>"}` schema.
pub fn ref_def(name: &str) -> Self {
let mut object = Map::new();
object.insert(
"$ref".to_string(),
Value::String(format!("#/$defs/{name}")),
);
Self { object }
}
// -----------------------------------------------------------------
// Annotation setters (ADR-003)
// -----------------------------------------------------------------
/// Schema-level endianness. Default little.
pub fn endian(mut self, endian: Endian) -> Self {
self.object.insert(
"endian".to_string(),
Value::String(
match endian {
Endian::Little => "little",
Endian::Big => "big",
}
.to_string(),
),
);
self
}
/// Struct or field alignment.
pub fn align(mut self, align: usize) -> Self {
self.object.insert("align".to_string(), Value::from(align as u64));
self
}
/// Variable-length encoding strategy.
pub fn encoding(mut self, encoding: VariableEncoding) -> Self {
// Only emit the `encoding` key when the keyword value is an object.
// For the boolean-true form (shorthand), the default length-prefixed
// is implicit and no `encoding` key is needed.
let kind_key = self
.object
.keys()
.find(|k| k.starts_with(ALKTYPE_PREFIX))
.cloned();
if let Some(k) = kind_key {
match encoding {
VariableEncoding::LengthPrefixed => {
// Default — make it explicit only if the keyword value is
// already an object. Otherwise leave the boolean form alone.
if let Some(Value::Object(_)) = self.object.get(&k) {
// The keyword value is an object; set encoding inside it.
// For the POC we don't reach this branch (boolean form only).
}
}
VariableEncoding::OffsetIndirect => {
// Replace the boolean-true form with the object form
// carrying the encoding annotation.
self.object.insert(k, Value::Object(Map::from_iter([
("encoding".to_string(), Value::String("offset-indirect".to_string())),
])));
}
}
}
self
}
/// `maxLength` — standard JSON Schema keyword. In aligned mode with a
/// variable-length type, reserves this many bytes (strategy 2). In
/// packed mode, validation constraint only.
pub fn max_length(mut self, max: usize) -> Self {
self.object.insert("maxLength".to_string(), Value::from(max as u64));
self
}
// -----------------------------------------------------------------
// Composite builders
// -----------------------------------------------------------------
/// Add a field to a struct or object. Field order is load-bearing
/// for binary layouts (packed mode field order = byte order).
pub fn field(mut self, name: &str, field: Schema) -> Self {
let props = self
.object
.entry("properties".to_string())
.or_insert_with(|| Value::Object(Map::new()))
.as_object_mut()
.expect("properties must be an object");
props.insert(name.to_string(), field.build());
self
}
/// Mark fields as required (standard JSON Schema `required` keyword).
/// Can be called multiple times; required names accumulate.
pub fn required(mut self, names: &[&str]) -> Self {
let entry = self
.object
.entry("required".to_string())
.or_insert_with(|| Value::Array(Vec::new()));
let arr = entry.as_array_mut().expect("required must be an array");
for n in names {
arr.push(Value::String((*n).to_string()));
}
self
}
/// Set the items schema for a standard `array` type.
pub fn items(mut self, item: Schema) -> Self {
self.object.insert("items".to_string(), item.build());
self
}
/// Set `additionalProperties` for a standard `object` type.
pub fn additional_properties(mut self, props: Schema) -> Self {
self.object
.insert("additionalProperties".to_string(), props.build());
self
}
/// Add a variant to a union. `disc_value` is the stringified
/// discriminator value (mapping key).
pub fn mapping(mut self, disc_value: &str, variant: Schema) -> Self {
let mapping = self
.object
.entry("mapping".to_string())
.or_insert_with(|| Value::Object(Map::new()))
.as_object_mut()
.expect("mapping must be an object");
mapping.insert(disc_value.to_string(), variant.build());
self
}
// -----------------------------------------------------------------
// Constraint setters (standard JSON Schema)
// -----------------------------------------------------------------
/// `minimum` (inclusive lower bound for numbers/integers).
pub fn minimum(mut self, min: f64) -> Self {
self.object.insert("minimum".to_string(), serde_json::Number::from_f64(min)
.map(Value::Number)
.unwrap_or(Value::Null));
self
}
/// `maximum` (inclusive upper bound for numbers/integers).
pub fn maximum(mut self, max: f64) -> Self {
self.object.insert("maximum".to_string(), serde_json::Number::from_f64(max)
.map(Value::Number)
.unwrap_or(Value::Null));
self
}
/// `minLength` (minimum string length).
pub fn min_length(mut self, min: usize) -> Self {
self.object.insert("minLength".to_string(), Value::from(min as u64));
self
}
/// `minItems` (minimum array length).
pub fn min_items(mut self, min: usize) -> Self {
self.object.insert("minItems".to_string(), Value::from(min as u64));
self
}
/// `maxItems` (maximum array length).
pub fn max_items(mut self, max: usize) -> Self {
self.object.insert("maxItems".to_string(), Value::from(max as u64));
self
}
/// `format` (e.g. "date-time", "uri", "email").
pub fn format(mut self, fmt: &str) -> Self {
self.object.insert("format".to_string(), Value::String(fmt.to_string()));
self
}
/// `title` (human-readable title).
pub fn title(mut self, t: &str) -> Self {
self.object.insert("title".to_string(), Value::String(t.to_string()));
self
}
/// `description` (human-readable description).
pub fn description(mut self, d: &str) -> Self {
self.object.insert("description".to_string(), Value::String(d.to_string()));
self
}
// -----------------------------------------------------------------
// Build
// -----------------------------------------------------------------
/// Produce the final `serde_json::Value`.
pub fn build(self) -> Value {
Value::Object(self.object)
}
}
/// TUnion discriminator, builder-friendly form. Mirrors
/// [`DiscriminatorKind`] but constructed via the builder API.
#[derive(Debug, Clone)]
pub enum Discriminator {
/// Byte-offset discriminator. `offset` is the byte position; `disc_type`
/// is the AlkType kind of the discriminator (Uint8/Uint16/Uint32).
Byte {
/// Byte position of the discriminator within the union's buffer.
offset: usize,
/// The AlkType kind of the discriminator (typically Uint8).
disc_type: AlkTypeKind,
},
/// Field-name discriminator. `name` is the field holding the
/// discriminator value.
Field {
/// The field name that holds the discriminator value.
name: String,
},
}
/// Helper for building named `$defs` that schemas can `$ref` by name.
///
/// `define` returns a `Schema` (the `$ref` to the definition), so it can
/// be passed directly to `.field()` / `.mapping()` / `.items()` without
/// a separate `ref_def` call. Call `.build()` once at the end to get the
/// `{"$defs": { ... }}` object to merge into a top-level schema.
#[derive(Debug, Default)]
pub struct Definitions {
defs: Map<String, Value>,
}
impl Definitions {
/// Create an empty `Definitions` registry.
pub fn new() -> Self {
Self::default()
}
/// Define a named schema. Returns a `Schema` that produces
/// `{"$ref": "#/$defs/<name>"}`.
pub fn define(&mut self, name: &str, schema: Schema) -> Schema {
self.defs.insert(name.to_string(), schema.build());
Schema::ref_def(name)
}
/// Like `define`, but the schema is an existing `Value` (adopted via
/// `Schema::from_value`).
pub fn define_value(&mut self, name: &str, value: Value) -> Schema {
self.defs.insert(name.to_string(), value);
Schema::ref_def(name)
}
/// Produce the `{"$defs": { ... }}` object to merge into a top-level
/// schema.
pub fn build(self) -> Value {
Value::Object(Map::from_iter([("$defs".to_string(), Value::Object(self.defs))]))
}
/// Merge the `$defs` into a top-level schema `Value`. Convenience
/// for the common case of building a schema that references its
/// definitions.
pub fn merge_into(self, top: &mut Value) {
if let Some(obj) = top.as_object_mut() {
if let Some(defs) = self.build().as_object() {
if let Some(existing) = obj.get_mut("$defs").and_then(Value::as_object_mut) {
existing.extend(defs.get("$defs").cloned().unwrap_or_default().as_object().cloned().unwrap_or_default());
} else {
obj.insert("$defs".to_string(), Value::Object(defs.get("$defs").cloned().unwrap_or_default().as_object().cloned().unwrap_or_default()));
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn builder_uint32_produces_keyword() {
assert_eq!(Schema::uint32().build(), json!({"AlkType:Uint32": true}));
}
#[test]
fn builder_struct_with_endian_and_fields() {
let s = Schema::struct_()
.endian(Endian::Big)
.field("channel_id", Schema::uint32())
.field("length", Schema::uint32())
.build();
assert_eq!(
s,
json!({
"AlkType:Struct": true,
"endian": "big",
"properties": {
"channel_id": {"AlkType:Uint32": true},
"length": {"AlkType:Uint32": true},
}
})
);
}
#[test]
fn builder_field_order_preserved() {
let s = Schema::struct_()
.field("first", Schema::uint8())
.field("second", Schema::uint16())
.field("third", Schema::uint32())
.build();
let props = s["properties"].as_object().unwrap();
let keys: Vec<&String> = props.keys().collect();
assert_eq!(keys, vec!["first", "second", "third"]);
}
#[test]
fn builder_object_with_required_and_constraints() {
let s = Schema::object()
.field("path", Schema::string_().max_length(4096))
.field("offset", Schema::integer().minimum(0.0))
.field("length", Schema::integer().minimum(0.0))
.required(&["path"])
.build();
assert_eq!(
s,
json!({
"type": "object",
"properties": {
"path": {"type": "string", "maxLength": 4096},
"offset": {"type": "integer", "minimum": 0.0},
"length": {"type": "integer", "minimum": 0.0}
},
"required": ["path"]
})
);
}
#[test]
fn builder_required_accumulates_across_calls() {
let s = Schema::object()
.field("a", Schema::integer())
.field("b", Schema::integer())
.required(&["a"])
.required(&["b"])
.build();
assert_eq!(s["required"], json!(["a", "b"]));
}
#[test]
fn builder_union_byte_discriminator_with_mapping() {
let s = Schema::union_(Discriminator::Byte {
offset: 0,
disc_type: AlkTypeKind::Uint8,
})
.mapping("5", Schema::ref_def("Read"))
.mapping("6", Schema::ref_def("Write"))
.build();
assert_eq!(
s,
json!({
"AlkType:Union": true,
"discriminator": {"kind": "byte", "offset": 0, "type": "AlkType:Uint8"},
"mapping": {
"5": {"$ref": "#/$defs/Read"},
"6": {"$ref": "#/$defs/Write"},
}
})
);
}
#[test]
fn builder_union_field_discriminator() {
let s = Schema::union_(Discriminator::Field { name: "type".to_string() })
.mapping("read", Schema::ref_def("Read"))
.build();
assert_eq!(
s["discriminator"],
json!({"kind": "field", "name": "type"})
);
}
#[test]
fn builder_enum_of_with_values() {
let s = Schema::enum_of(&["read", "write", "execute"]).build();
assert_eq!(
s,
json!({
"AlkType:Enum": true,
"enum": ["read", "write", "execute"]
})
);
}
#[test]
fn builder_array_of_and_record_of() {
let arr = Schema::array_of(Schema::uint32()).build();
assert_eq!(
arr,
json!({"AlkType:Array": true, "items": {"AlkType:Uint32": true}})
);
let rec = Schema::record_of(Schema::float32()).build();
assert_eq!(
rec,
json!({"AlkType:Record": true, "values": {"AlkType:Float32": true}})
);
}
#[test]
fn builder_definitions_define_returns_ref() {
let mut defs = Definitions::new();
let r = defs.define("FileNotFound", Schema::object().field("path", Schema::string_()));
assert_eq!(r.build(), json!({"$ref": "#/$defs/FileNotFound"}));
let built = defs.build();
assert_eq!(
built["$defs"]["FileNotFound"],
json!({"type": "object", "properties": {"path": {"type": "string"}}})
);
}
#[test]
fn builder_ref_def_produces_json_pointer() {
assert_eq!(
Schema::ref_def("Read").build(),
json!({"$ref": "#/$defs/Read"})
);
}
#[test]
fn builder_encoding_offset_indirect_rewrites_to_object_form() {
let s = Schema::string().encoding(VariableEncoding::OffsetIndirect).build();
assert_eq!(
s,
json!({"AlkType:String": {"encoding": "offset-indirect"}})
);
}
#[test]
fn builder_align_sets_annotation() {
let s = Schema::struct_().align(256).build();
assert_eq!(s["align"], json!(256));
}
#[test]
fn builder_from_value_adopts_existing_schema() {
let existing = json!({"type": "integer", "minimum": 0});
let s = Schema::from_value(existing.clone());
assert_eq!(s.build(), existing);
}
#[test]
fn builder_any_is_empty_object() {
assert_eq!(Schema::any().build(), json!({}));
}
#[test]
fn builder_chunk_header_compiles_in_packed_mode() {
// The POC's primary use case: channels' 8-byte chunk header.
let mut schema = Schema::struct_()
.endian(Endian::Big)
.field("channel_id", Schema::uint32())
.field("length", Schema::uint32())
.build();
let engine = crate::AlkTypeEngine::compile(&mut schema, crate::LayoutMode::Packed);
assert!(engine.is_ok(), "chunk header schema should compile: {engine:?}");
}
}