Files
alktype/docs/architecture/builder.md
glm-5.2 c0217d91a8 Resolve v0.1.0 open questions and fix production-readiness issues
POC: /workspace/alktype-builder-poc/ (18/18 tests pass, findings in
docs/research/alktype-builder-poc/findings.md). Round 2 adds the SFTP
Packet validate_bytes tests (7 new: valid Init/Read/Write/Status,
short buffer, unknown discriminator, over-maxLength Bytes).

Open questions resolved (OQ-004 through OQ-008):
- OQ-004: Discriminator::Field name is String (already implemented;
  docs updated to mark resolved)
- OQ-005: Both union discriminator kinds return the same shape:
  {__discriminator, ...variant-fields}. Field-name path also had a
  real offset bug (returned start, not end) - fixed.
- OQ-006: builder.md Example 3 now wraps the Union in a
  Schema::struct_().field("payload", ...) and merges $defs via
  Definitions::merge_into (matches the engine's AlkType:Struct-at-root
  constraint and the SFTP wire shape)
- OQ-007: Bytes materialization is array-of-u8 (Value::Array of
  Value::Number, one entry per byte 0..=255). BytesValidator accepts
  both Value::String (validate_json) and Value::Array (validate_bytes).
  maxLength = max byte count. Replaces the lossy from_utf8_lossy path
  that corrupted non-UTF-8 bytes and broke maxLength semantics.
- OQ-008 (new): UnionValidator now dispatches to variant schemas via
  sub-validators built at factory time. AlkTypeEngine::compile calls
  schema::inline_union_variant_refs before build_validator to inline
  $refs in union mapping entries (necessary because union_factory
  receives the union node, but $defs live at the schema root).

Production-readiness fixes in src/ (no stubs/hedges in a published crate):
- materialize.rs: Record stub -> full count-prefixed key/value pair
  implementation per schema-layer.md TRecord
- materialize.rs: root_of() was broken (returned the current node, not
  the schema root) -> root schema threaded through every recursive call
  so resolve_ref_or_inline can resolve $refs for nested composites
- builder.rs: LengthPrefixed encoding setter was a no-op when the
  keyword was already in object form -> complete the branch (updates
  the encoding entry in place for both LengthPrefixed and OffsetIndirect)
- builder.rs, engine.rs: POC-referencing comments cleaned up; the
  round-trip test's or_else fallback (papering over write_field being
  aligned-only) replaced with direct byte writes

Documentation:
- builder.md: Example 3 updated; Discriminator::Field spec shows String;
  Open Questions section updated (OQ-004 resolved)
- validation.md: AlkType:Bytes and AlkType:Union validator descriptions
  updated for array-of-u8 form and variant dispatch
- open-questions.md: OQ-004/005/006/007/008 marked resolved; new
  Validation theme entries
- questions/004-008: individual OQ files updated with resolutions
- findings.md: round 2 results documented

Verification:
- cargo test: 369 -> 391 tests pass (22 new: 14 materialize, 5
  inline_union_variant_refs, 3 validation/builder)
- cargo clippy --all-targets: clean
- POC: 11 -> 18 tests (7 new SFTP Packet tests); all pass
2026-08-11 07:28:24 +00:00

22 KiB

status, last_updated
status last_updated
draft 2026-08-11

alktype — Builder API

The builder layer: a fluent Rust API for constructing alktype JSON Schemas (both AlkType:*-bearing binary-layout schemas and plain JSON-Schema-only operation payload schemas) at runtime, producing serde_json::Value. Decided in ADR-009; resolves OQ-003.

What

The builder module provides a single Schema builder type and a Definitions helper for named $defs. The builder's .build() method returns a serde_json::Value — the same form alktype already consumes via AlkTypeEngine::compile (for AlkType:* schemas) and the same form OperationSpec.input_schema / output_schema / error_schemas hold (for plain JSON Schema, no AlkType:* kinds).

The builder covers:

  • All 19 AlkType:* kinds (binary-layout schemas) — see schema-layer.md for the kinds.
  • All standard JSON Schema keywords needed for operation payload schemas: type, properties, required, items, enum, format, additionalProperties, minimum, maximum, minItems, maxItems, minLength, maxLength, $ref, $defs.
  • All ADR-003 schema annotations: endian, align, encoding (length-prefixed / offset-indirect), maxLength-as-reservation, and TUnion discriminator (byte-offset and field-name).

Why

alkcall (the merged alknet-call + alknet-channels extraction) is alktype's first consumer and needs to build schemas at runtime from Rust code, for two roles:

  1. Binary layout schemas (channels' 8-byte chunk header, future binary call frames) — AlkType:* schemas, fed to AlkTypeEngine::compile (packed mode, big-endian).
  2. JSON payload schemas (call's OperationSpec.input_schema / output_schema / error_schemas) — plain JSON Schema, no AlkType:* kinds, validated via the standard jsonschema validator.

A single builder serving both roles means alkcall imports one module for schema construction. See ADR-009 for the decision rationale (why Value not a typed Schema enum, why both AlkType and standard JSON Schema in one builder).

Architecture

Output type: serde_json::Value

The builder returns serde_json::Value. There is no typed Schema enum. This matches ADR-001's "schemas are JSON" principle and avoids duplicating the JSON form that AlkTypeEngine::compile, jsonschema::Validator, and OperationSpec all consume.

Module placement

src/builder.rs, re-exported from the crate root. The builder is a peer of schema.rs (which parses schemas) and engine.rs (which compiles them). The builder constructs; it does not parse or compile.

// src/lib.rs (additions)
pub mod builder;
pub use builder::{Schema, Definitions};

Field order is load-bearing

serde_json with preserve_order is already a dependency (ADR-001). The builder's Value output uses serde_json::Map (which preserves insertion order under preserve_order), so field declaration order in the builder is the field order in the binary layout. This is critical for packed mode (ADR-002) where field order determines offsets.

Public API

Schema builder

Schema is the single entry point. Constructors for each AlkType kind and each standard JSON Schema type; setters for annotations and constraints; .build() produces Value.

AlkType kind constructors

One constructor per AlkTypeKind variant (see schema-layer.md §"The 19 AlkType Kinds"):

impl Schema {
    // Fixed-size integer kinds
    pub fn int8()    -> Self;
    pub fn int16()   -> Self;
    pub fn int32()   -> Self;
    pub fn int64()   -> Self;
    pub fn uint8()   -> Self;
    pub fn uint16()  -> Self;
    pub fn uint32()  -> Self;
    pub fn uint64()  -> Self;
    // Float kinds
    pub fn float32() -> Self;
    pub fn float64() -> Self;
    // Other fixed-size
    pub fn boolean() -> Self;
    pub fn enum_of(values: &[&str]) -> Self;  // u32 index; values in declaration order
    // Variable-length kinds
    pub fn string()    -> Self;
    pub fn bytes()     -> Self;
    pub fn timestamp() -> Self;
    // Composite kinds
    pub fn struct_() -> Self;  // fields added via .field()
    pub fn union_(disc: Discriminator) -> Self;  // variants via .mapping()
    pub fn array_of(element: Schema) -> Self;
    pub fn record_of(value: Schema) -> Self;
}

Each constructor sets the corresponding "AlkType:<Kind>": true key. For example, Schema::uint32() produces {"AlkType:Uint32": true}.

enum_of sets both "AlkType:Enum": true and the standard "enum" keyword with the provided values (declaration order is the index order — see schema-layer.md §"TEnum binary representation"):

Schema::enum_of(&["read", "write", "execute"])
// -> { "AlkType:Enum": true, "enum": ["read", "write", "execute"] }

array_of and record_of take the element/value schema as a nested Schema:

Schema::array_of(Schema::uint32())
// -> { "AlkType:Array": true, "items": { "AlkType:Uint32": true } }

Schema::record_of(Schema::float32())
// -> { "AlkType:Record": true, "values": { "AlkType:Float32": true } }

Standard JSON Schema type constructors

For plain JSON Schema (no AlkType:* kinds) — call's input_schema / output_schema / error_schemas:

impl Schema {
    pub fn object()  -> Self;  // type: "object" — fields via .field()
    pub fn array()   -> Self;  // type: "array"  — items via .items()
    pub fn string_() -> Self;  // type: "string"
    pub fn integer() -> Self;  // type: "integer"
    pub fn number()  -> Self;  // type: "number"
    pub fn boolean_()-> Self;  // type: "boolean"
    pub fn null()    -> Self;  // type: "null"
    pub fn any()     -> Self;  // {} (no type constraint)
}

The _ suffix disambiguates standard JSON Schema types from AlkType kinds (string is the AlkType kind; string_ is the standard JSON Schema type — the AlkType kind constructor sets "AlkType:String": true, the standard constructor sets "type": "string"). This is deliberate: the two are distinct schema forms and the builder makes the distinction visible at the call site.

Annotation setters

Annotation setters mirror ADR-003. Each setter is named after the annotation it produces; calling the setter sets the corresponding JSON key. Setters return Self for chaining.

impl Schema {
    /// Schema-level endianness (ADR-003 §1). Default little.
    pub fn endian(mut self, endian: Endian) -> Self;

    /// Struct or field alignment (ADR-003 §2). Struct-level sets the
    /// default; field-level overrides.
    pub fn align(mut self, align: usize) -> Self;

    /// Variable-length encoding strategy (ADR-003 §3).
    /// Default length-prefixed.
    pub fn encoding(mut self, encoding: VariableEncoding) -> Self;

    /// maxLength — standard JSON Schema keyword. In aligned mode with
    /// a variable-length type, reserves this many bytes (strategy 2).
    /// In packed mode, validation constraint only.
    pub fn max_length(mut self, max: usize) -> Self;
}

Endian and VariableEncoding are re-exported from schema.rs (no new types — the builder uses the existing enums). The setters produce the exact JSON shapes from ADR-003:

Schema::struct_()
    .endian(Endian::Big)
    .field("channel_id", Schema::uint32())
    .field("length",     Schema::uint32())
    .build()
// -> {
//      "AlkType:Struct": true,
//      "endian": "big",
//      "properties": {
//        "channel_id": { "AlkType:Uint32": true },
//        "length":     { "AlkType:Uint32": true }
//      }
//    }

Composite builders

struct_(), union_(), array_of(), record_of() are the composite constructors. struct_() and union_() need additional setters to populate their children:

impl Schema {
    /// Add a field to a struct (or object). Field order is load-bearing
    /// for binary layouts (packed mode field order = byte order).
    /// The field's schema is built from the passed `Schema`.
    pub fn field(mut self, name: &str, field: Schema) -> Self;

    /// Mark fields as required (standard JSON Schema `required` keyword).
    /// Can be called multiple times; required names accumulate.
    /// Field names must have been added via `.field()`.
    pub fn required(mut self, names: &[&str]) -> Self;

    /// Set the items schema for a standard `array` type.
    pub fn items(mut self, item: Schema) -> Self;

    /// Set the additionalProperties schema for a standard `object` type.
    pub fn additional_properties(mut self, props: Schema) -> Self;

    /// Add a variant to a union. `disc_value` is the stringified
    /// discriminator value (mapping key). The variant schema is built
    /// from the passed `Schema`.
    pub fn mapping(mut self, disc_value: &str, variant: Schema) -> Self;
}

field sets properties[name] = field.build(). Repeated calls append. Field order in the built Value is the call order (because serde_json::Map preserves insertion order under preserve_order).

required sets the standard JSON Schema "required" array. The builder does not check that the named fields exist (that's a compile-time check at AlkTypeEngine::compile, per ADR-004's load-time validation strategy — see ADR-009 §"What the builder is not"). Calling required multiple times accumulates names:

Schema::object()
    .field("path",   Schema::string_())
    .field("offset", Schema::integer().minimum(0))
    .field("length", Schema::integer().minimum(0))
    .required(["path"])
    .required(["offset", "length"])
// -> {
//      "type": "object",
//      "properties": { "path": {...}, "offset": {...}, "length": {...} },
//      "required": ["path", "offset", "length"]
//    }

Constraint setters (standard JSON Schema)

For operation payload schemas (call's input_schema etc.):

impl Schema {
    /// `minimum` (inclusive lower bound for numbers/integers).
    pub fn minimum(mut self, min: f64) -> Self;

    /// `maximum` (inclusive upper bound for numbers/integers).
    pub fn maximum(mut self, max: f64) -> Self;

    /// `minLength` (minimum string length).
    pub fn min_length(mut self, min: usize) -> Self;

    /// `minItems` (minimum array length).
    pub fn min_items(mut self, min: usize) -> Self;

    /// `maxItems` (maximum array length).
    pub fn max_items(mut self, max: usize) -> Self;

    /// `format` (e.g. "date-time", "uri", "email").
    pub fn format(mut self, fmt: &str) -> Self;

    /// `title` (human-readable description).
    pub fn title(mut self, t: &str) -> Self;

    /// `description` (human-readable description).
    pub fn description(mut self, d: &str) -> Self;
}

These set the corresponding standard JSON Schema keywords. They apply to both AlkType-kind schemas and standard JSON Schema type schemas (e.g., Schema::string().max_length(4096) sets maxLength, which serves as both a validation constraint and, in aligned mode, a fixed-size reservation — ADR-003 §3).

.build()

impl Schema {
    /// Produce the final `serde_json::Value`.
    pub fn build(self) -> Value;
}

Consumes the builder and returns the assembled Value. The builder is not Clone (to discourage partial builds); each Schema is consumed once. To reuse a sub-schema, build it once and pass the Value to a Schema::from_value constructor (below).

Schema::from_value — adopt an existing Value

impl Schema {
    /// Adopt an existing JSON Schema `Value` as a `Schema`, for
    /// composition with builder-constructed schemas. Does not validate
    /// the schema; just wraps it.
    pub fn from_value(value: Value) -> Self;
}

For consumers that have some schemas as JSON (e.g., loaded from a TypeBox-produced file) and others built via the fluent API, and want to compose them. from_value wraps the Value so it can be passed to .field() / .items() / .mapping() like any other Schema.

Discriminator for union_()

union_() takes a Discriminator describing the union's dispatch mechanism. This mirrors schema.rs::DiscriminatorKind but with a builder-friendly shape (the kind enum is re-exported from schema.rs, not duplicated):

pub enum Discriminator {
    /// Byte-offset discriminator (ADR-003 §4 Kind A).
    /// `offset` is the byte position; `disc_type` is the AlkType kind
    /// of the discriminator (Uint8/Uint16/Uint32).
    Byte {
        offset: usize,
        disc_type: AlkTypeKind,  // restricted to Uint8/Uint16/Uint32
    },
    /// Field-name discriminator (ADR-003 §4 Kind B).
    /// `name` is the field holding the discriminator value.
    Field {
        name: String,
    },
}

Byte-offset example (channels-style protocol dispatch):

let packet = Schema::union_(Discriminator::Byte {
        offset: 0,
        disc_type: AlkTypeKind::Uint8,
    })
    .mapping("5",   Schema::ref_def("Read"))
    .mapping("6",   Schema::ref_def("Write"))
    .mapping("101", Schema::ref_def("Status"))
    .build();
// -> {
//      "AlkType:Union": true,
//      "discriminator": { "kind": "byte", "offset": 0, "type": "AlkType:Uint8" },
//      "mapping": { "5": {"$ref":"#/$defs/Read"}, "6": {...}, "101": {...} }
//    }

Field-name example (typedef.ts pattern):

let event = Schema::union_(Discriminator::Field { name: "type" })
    .mapping("read",  Schema::ref_def("Read"))
    .mapping("write", Schema::ref_def("Write"))
    .build();
// -> {
//      "AlkType:Union": true,
//      "discriminator": { "kind": "field", "name": "type" },
//      "mapping": { "read": {...}, "write": {...} }
//    }

Definitions — named $defs for cross-reference

Definitions is a helper for building named $defs that schemas can $ref by name. This is the ergonomics win for alkcall's OperationSpec, where input/output/error schemas reference shared definitions (e.g., FileNotFound, RateLimited).

pub struct Definitions { /* ... */ }

impl Definitions {
    pub fn new() -> Self;

    /// Define a named schema. Returns a `Schema` that produces
    /// `{"$ref": "#/$defs/<name>"}` — the JSON Pointer form that
    /// `jsonschema` and `AlkTypeEngine::compile` expect (after
    /// `normalize_refs`, which the engine runs at compile time).
    pub fn define(&mut self, name: &str, schema: Schema) -> Schema;

    /// Like `define`, but the schema is an existing `Value` (adopted
    /// via `Schema::from_value`).
    pub fn define_value(&mut self, name: &str, value: Value) -> Schema;

    /// Produce the `{"$defs": { ... }}` object to merge into a
    /// top-level schema. Call once at the end.
    pub fn build(self) -> Value;
}

Usage:

let mut defs = Definitions::new();

let file_not_found = defs.define("FileNotFound",
    Schema::object()
        .field("path",  Schema::string_())
        .field("errno",  Schema::integer())
        .required(["path", "errno"])
);

let rate_limited = defs.define("RateLimited",
    Schema::object()
        .field("retry_after_ms", Schema::integer().minimum(0))
        .required(["retry_after_ms"])
);

let read_file_error = Schema::object()
    .field("code", Schema::string_())
    .field("details", Schema::any())  // one of the defined errors
    .required(["code"])
    .build();

// Merge $defs into the top-level schema that references them
let mut top = Schema::object()
    .field("error", read_file_error)
    .build();
top.as_object_mut().unwrap().insert("$defs".to_string(), defs.build());

define returns a Schema (the $ref to the definition), so it can be passed directly to .field() / .mapping() / .items() without a separate ref_def call:

let op = Schema::object()
    .field("error", defs.define("FileNotFound", /* ... */))
    .build();

Schema::ref_def — reference a definition by name

impl Schema {
    /// Produce a `{"$ref": "#/$defs/<name>"}` schema. The definition
    /// must exist in the `$defs` of the top-level schema at compile
    /// time. The builder does not check this.
    pub fn ref_def(name: &str) -> Self;
}

For cases where the Definitions::define return value isn't handy (e.g., referencing a definition defined elsewhere). Produces the same {"$ref": "#/$defs/<name>"} form.

Usage Examples

Example 1: channels' 8-byte chunk header (binary layout)

use alktype::{Schema, Endian};

let chunk_header = Schema::struct_()
    .endian(Endian::Big)
    .field("channel_id", Schema::uint32())
    .field("length",     Schema::uint32())
    .build();

// -> {
//      "AlkType:Struct": true,
//      "endian": "big",
//      "properties": {
//        "channel_id": { "AlkType:Uint32": true },
//        "length":     { "AlkType:Uint32": true }
//      }
//    }
//
// Feed to AlkTypeEngine::compile(&mut chunk_header, LayoutMode::Packed)
// then validate incoming frames via engine.validate_bytes(&frame).

Example 2: call's OperationSpec input schema (JSON payload)

use alktype::Schema;

let read_file_input = Schema::object()
    .field("path",   Schema::string_().max_length(4096))
    .field("offset", Schema::integer().minimum(0))
    .field("length", Schema::integer().minimum(0))
    .required(["path"])
    .build();

// -> {
//      "type": "object",
//      "properties": {
//        "path":   { "type": "string", "maxLength": 4096 },
//        "offset": { "type": "integer", "minimum": 0 },
//        "length": { "type": "integer", "minimum": 0 }
//      },
//      "required": ["path"]
//    }
//
// Stored in OperationSpec.input_schema; validated via the standard
// jsonschema validator (validate_json for parsed payloads, or via
// serde_json::from_slice then validate_json for wire frames).

Example 3: SFTP Packet union (binary layout, byte discriminator)

The SFTP wire shape is [type:u8][payload-struct] — a struct with a union payload field. The engine requires AlkType:Struct at the top level (OffsetMap::compute / SequentialReader::new both enforce this; a Union is a field type within a struct, not a top-level schema). The builder constructs the union wrapped in a struct, and $defs are merged into the top-level schema so $refs resolve:

use alktype::{Definitions, Discriminator, AlkTypeKind, Schema};

let mut defs = Definitions::new();
defs.define("Init",  Schema::struct_().field("version", Schema::uint32()));
defs.define("Open",  Schema::struct_().field("path", Schema::string()).field("flags", Schema::uint32()));
defs.define("Read",  Schema::struct_().field("handle", Schema::bytes()).field("offset", Schema::uint64()).field("len", Schema::uint32()));
defs.define("Write", Schema::struct_().field("handle", Schema::bytes()).field("offset", Schema::uint64()).field("data", Schema::bytes()));
defs.define("Status", Schema::struct_().field("code", Schema::uint32()).field("message", Schema::string()));

// A "Packet" is a struct with one field — the union. This mirrors
// SFTP's wire shape: [type:u8][payload-struct].
let mut packet = Schema::struct_()
    .field(
        "payload",
        Schema::union_(Discriminator::Byte {
            offset: 0,
            disc_type: AlkTypeKind::Uint8,
        })
        .mapping("1",   Schema::ref_def("Init"))
        .mapping("3",   Schema::ref_def("Open"))
        .mapping("5",   Schema::ref_def("Read"))
        .mapping("6",   Schema::ref_def("Write"))
        .mapping("101", Schema::ref_def("Status")),
    )
    .build();
// Merge $defs into the top-level schema so $refs resolve at compile time.
defs.merge_into(&mut packet);
// Feed to AlkTypeEngine::compile(&mut packet, LayoutMode::Packed)
// then validate incoming frames via engine.validate_bytes(&frame).

Example 4: OperationSpec error schemas (named $defs)

use alktype::{Definitions, Schema};

let mut defs = Definitions::new();

let file_not_found = defs.define("FileNotFound",
    Schema::object()
        .field("path",  Schema::string_())
        .field("errno", Schema::integer())
        .required(["path", "errno"])
);

let rate_limited = defs.define("RateLimited",
    Schema::object()
        .field("retry_after_ms", Schema::integer().minimum(0))
        .required(["retry_after_ms"])
);

// An operation's error schemas reference these definitions
let op_errors = vec![
    ErrorDefinition {
        code: "FILE_NOT_FOUND".to_string(),
        description: "File not found".to_string(),
        schema: Schema::ref_def("FileNotFound").build(),
        http_status: Some(404),
    },
    ErrorDefinition {
        code: "RATE_LIMITED".to_string(),
        description: "Rate limited".to_string(),
        schema: Schema::ref_def("RateLimited").build(),
        http_status: Some(429),
    },
];
// $defs is built once and stored alongside the OperationSpec

Design Decisions

Decision ADR Summary
Builder API for schema construction ADR-009 Fluent Rust API producing serde_json::Value; covers AlkType kinds + standard JSON Schema; resolves OQ-003
Schema annotations ADR-003 The annotation shapes the builder's setters produce
Load-time validation strategy ADR-004 The builder does not pre-validate; compile-time is the validation point

Open Questions

None specific to the builder. OQ-003 (the original "should we build a builder API") is resolved by this spec / ADR-009. OQ-004 (Discriminator::Field name type — &str or String) is resolved: String, for ownership simplicity (the builder consumes Self on setters; &str would require a lifetime parameter on Discriminator and transitively on Schema::union_). See open-questions.md.

References

  • ADR-009 — the decision this spec implements
  • ADR-001 — scope boundaries this module extends; "schemas are JSON" principle
  • ADR-003 — the annotation shapes the builder's setters produce
  • schema-layer.md — the 19 AlkType kinds the builder's constructors produce
  • validation.md — the validation layer that consumes builder output (via AlkTypeEngine::compile)
  • @alkdev/alknet: docs/architecture/crates/call/operation-registry.mdOperationSpec (the alkcall consumer for the JSON-payload role)
  • @alkdev/alknet: docs/architecture/crates/channels/channels-wire.md — the 8-byte chunk header (the alkcall consumer for the binary-layout role)