alktype/docs/architecture/validation.md

status, last_updated

status	last_updated
draft	2026-08-11

alktype — Validation

The validation layer: custom keyword validators for all 19 AlkType:* kinds, the AlkTypeError enum, load-time vs access-time validation strategy, and the AlkTypeEngine as the compiled form of a schema.

Validation Strategy

Validation is delegated to the jsonschema crate (v0.46.5, Draft 2020-12). The alktype engine does not implement its own validation — it registers custom keyword validators for each AlkType:* kind and lets jsonschema handle the structural validation (object properties, required fields, array items, enum values).

The strategy is decided in ADR-004:

1. Load time: Parse the schema JSON, build the layout engine, build the jsonschema validator. This is the AlkTypeEngine::compile(schema) constructor.
2. Access time: Use the compiled engine for repeated read/write operations. Validation is opt-in per operation.

What validation validates

The jsonschema validator operates on serde_json::Value instances — it validates JSON representations of data, not raw byte buffers. This is the correct separation of concerns:

• JSON validation (jsonschema): validates that a JSON document conforms to the schema. Used for validating hand-written schemas, TypeBox output, JSON payloads, or the JSON representation of a binary struct after deserialization.
• Binary access validation (data access layer): the read/write functions perform type-level validation at access time — range checks for integers, UTF-8 validity for strings, buffer bounds checking. These return AlkTypeError::Access with field paths.

The "schema is the format" principle means the same schema describes both the JSON shape and the binary layout. The jsonschema validator checks the JSON shape; the data access layer checks the binary layout. A consumer that wants to validate a binary buffer end-to-end reads the buffer into a Value tree via the data access layer, then validates that Value against the jsonschema validator. This is a two-step process, not a single validate(buffer) call.

The AlkTypeEngine struct

The AlkTypeEngine is the compiled form of a schema. It supports both layout modes (ADR-002) via an internal Layout enum:

pub struct AlkTypeEngine {
    layout: Layout,                   // packed or aligned (private enum)
    validator: jsonschema::Validator, // compiled once at load time
    endian: Endian,                   // parsed from the schema's "endian" annotation
    schema: Value,                    // the normalized schema (refs resolved)
}

// Private — the consumer selects via LayoutMode at compile time.
enum Layout {
    Packed { builder: LayoutBuilder },
    Aligned { offset_map: OffsetMap },
}

The consumer selects the mode at construction time via LayoutMode (see layout-engine.md §"Mode Selection"). The Layout enum is private — the engine exposes mode-appropriate accessors instead:

impl AlkTypeEngine {
    pub fn compile(schema: &mut Value, mode: LayoutMode) -> Result<Self, AlkTypeError>;
    pub fn mode(&self) -> LayoutMode;
    pub fn endian(&self) -> Endian;
    pub fn offset_map(&self) -> Option<&OffsetMap>;          // Some in aligned mode
    pub fn layout_builder(&self) -> Option<&LayoutBuilder>;  // Some in packed mode
    pub fn sequential_reader(&self) -> Option<SequentialReader>; // owned fresh reader (ADR-007)
    pub fn validate_json(&self, instance: &Value) -> Result<(), AlkTypeError>;       // ADR-004
    pub fn is_valid_json(&self, instance: &Value) -> bool;                           // ADR-004
    pub fn validate_bytes(&self, buffer: &[u8]) -> Result<(), AlkTypeError>;        // ADR-010
}

compile takes &mut Value because it normalizes $ref values in place (via normalize_refs) before computing the layout and building the validator. The schema field retains the normalized schema for read_field's kind lookup and for sequential_reader()'s factory construction. The validator is mode-agnostic (it operates on Value, not raw bytes).

The Layout::Packed variant stores only the LayoutBuilder (write-side). The SequentialReader (read-side) is not stored — it has mutable cursor state that the consumer owns, so sequential_reader() constructs a fresh reader on each call (ADR-007).

The read_field/write_field methods on AlkTypeEngine are the aligned-mode data-access API — see data-access.md §"Higher-level read/write".

Custom Keyword Validators

Each AlkType:* kind gets a Keyword implementation registered via jsonschema::options().with_keyword(...). The validators check leaf type constraints; jsonschema handles all structural validation.

Numeric type validators

AlkType:Float32 / AlkType:Float64:

• Value must be a finite number.
• For Float32: value must be representable as f32 (no precision loss beyond f32's mantissa).

AlkType:Int8 / AlkType:Int16 / AlkType:Int32:

• Value must be an integer within the type's range.
• Int8: -128..127, Int16: -32768..32767, Int32: -2147483648..2147483647.

AlkType:Uint8 / AlkType:Uint16 / AlkType:Uint32:

• Value must be a non-negative integer within the type's range.
• Uint8: 0..255, Uint16: 0..65535, Uint32: 0..4294967295.

String and binary validators

AlkType:String:

• Value must be a valid UTF-8 string.
• If maxLength is specified in the schema, the string's byte length must not exceed it.

AlkType:Bytes:

• Value must be a string (the JSON form for validate_json consumers) or an array of integers 0..=255 (the materialized form for validate_bytes). JSON has no native byte type; the string form is the JSON convention, the array form is the round-trippable form for non-UTF-8 bytes (see OQ-007).
• If maxLength is specified, the byte length must not exceed it. For the string form, this is the string's byte length; for the array form, this is the array length (one entry per byte).
• Binary representation: In the binary layout, TBytes is raw bytes with no encoding (not base64, not hex). The JSON representation (for validation) uses a string or array; the binary representation (for data access) uses &[u8] directly.

AlkType:Enum:

• The AlkType:Enum custom keyword signals that the type is an enum for layout purposes (the engine needs to know it's a fixed-size u32 index, not a variable-length string). The built-in enum keyword provides the value list and handles value-membership validation. The custom keyword validator is a no-op beyond the built-in check — it exists solely for the layout engine to recognize the type.

AlkType:Timestamp:

• Value must be a valid RFC 3339 timestamp string (the internet profile of ISO 8601, e.g., "2026-07-20T15:30:00Z").

Composite type validators

AlkType:Struct:

• Value must be an object.
• Each property must match its declared AlkType:* kind.
• Required fields must be present.
• The jsonschema crate's built-in properties and required keywords handle the structural checks — the custom keyword only needs to validate that each field's value matches its AlkType:* kind.

AlkType:Union:

• The instance must be an object with a __discriminator field carrying the mapping key (stringified discriminator value for byte-offset discriminators, string value for field-name discriminators). This is the shape the materializer produces for validate_bytes; validate_json consumers produce the same shape when validating a union instance.
• The UnionValidator builds a sub-validator for each variant at factory time (when the parent validator tree is constructed) and dispatches on __discriminator at validation time, validating the full instance (including the variant fields) against the selected variant's schema. This closes the OQ-008 gap: variant field constraints (e.g. maxLength on a Bytes field inside a variant) are checked.
• $refs in the union's mapping are inlined by schema::inline_union_variant_refs during AlkTypeEngine::compile (before build_validator), so the union_factory sees full inline variant schemas. See OQ-008.

AlkType:Array:

• Value must be an array.
• Each element must match the array's declared element type.
• If minItems/maxItems is specified, the array length must be within bounds.

Other validators

AlkType:Boolean:

• Value must be true or false.

AlkType:Record:

• Value must be an object.
• All values must match the record's declared value type (specified via the "values" property in the schema, e.g., "values": { "AlkType:Float32": true }).

Validator implementation pattern

Each custom keyword implementation is ~10 lines. Example for AlkType:Float32:

struct Float32Validator;

impl Keyword for Float32Validator {
    fn validate<'i>(&self, instance: &'i Value) -> Result<(), ValidationError<'i>> {
        match instance {
            Value::Number(n) if n.as_f64().map_or(false, |f| f.is_finite()) => Ok(()),
            _ => Err(ValidationError::custom("expected finite f32-compatible number")),
        }
    }
    fn is_valid(&self, instance: &Value) -> bool {
        instance.as_f64().map_or(false, |f| f.is_finite())
    }
}

Registration:

let validator = jsonschema::options()
    .with_keyword("AlkType:Float32", |parent, value, path| {
        Ok(Box::new(Float32Validator))
    })
    .build(&schema)?;

The factory closure receives the parent schema object, the keyword's value, and the schema path. This enables cross-keyword awareness — for example, a AlkType:Struct validator can inspect the parent's properties to validate each field against its declared AlkType:* kind.

AlkTypeError

A single AlkTypeError enum covers all error conditions across the engine's three phases (schema parsing, offset computation, read/write) plus validation. Decided in ADR-004.

pub enum AlkTypeError {
    /// Schema parsing errors (invalid JSON, missing keywords, unknown AlkType kinds).
    Schema(String),
    /// Offset computation errors (field not found, unsupported type).
    Offset { field_path: String, reason: String },
    /// Read/write errors (buffer too short, invalid UTF-8, value out of range).
    Access { field_path: String, reason: String },
    /// Validation errors (delegated to jsonschema).
    Validation(ValidationError<'static>),
}

• Schema — for errors during AlkTypeEngine::compile(). Invalid JSON, missing required keywords, unknown AlkType:* kinds.
• Offset — for errors during offset computation. Field not found in the schema, type not supported for offset computation, recursive depth exceeded. Carries the field path.
• Access — for errors during read/write. Buffer too short, invalid UTF-8 in a string field, value out of range for the target type. Carries the field path.
• Validation — wraps jsonschema's ValidationError. The 'static lifetime is correct — the validator owns its schema reference and lives for the lifetime of the AlkTypeEngine.

Field-path-carrying errors

Read/write and offset errors include the field path for debugging:

Err(AlkTypeError::Access {
    field_path: "header.version".to_string(),
    reason: "buffer too short: need 4 bytes at offset 12, have 2".to_string(),
})

This makes debugging binary format issues tractable — the error tells you exactly which field failed and why.

Validation Timing

Load time: AlkTypeEngine::compile()

The expensive work happens once at schema load time:

1. Normalize $ref values in the schema (normalize_refs).
2. Parse the schema's "endian" annotation.
3. Compute the layout (LayoutBuilder/SequentialReader for packed, OffsetMap for aligned).
4. Build the jsonschema validator (jsonschema::options().with_keyword(...).build(&schema)?).

The result is a AlkTypeEngine that can be used for repeated operations.

Access time: engine.validate_json(&Value) / engine.is_valid_json(&Value)

Validation is opt-in per operation. The consumer calls engine.validate_json(instance) when validation is desired, or engine.is_valid_json(instance) for a boolean check. The jsonschema validator is already compiled — these are fast checks against the compiled validator.

pub fn validate_json(&self, instance: &Value) -> Result<(), AlkTypeError>;
pub fn is_valid_json(&self, instance: &Value) -> bool;

The argument is a serde_json::Value (the JSON representation of the data), not a raw byte buffer — see §"What validation validates" above. To validate a binary buffer end-to-end, the consumer reads it into a Value tree via the data access layer, then validates that Value.

High-throughput paths can skip validation. Security-sensitive paths (parsing incoming frames from untrusted peers) can validate every frame. The choice is the consumer's.

Access time: engine.validate_bytes(&[u8]) — binary buffer validation

For binary-layout schemas (schemas declaring AlkType:* kinds), the engine offers a single-call form of the two-step dance: walk the bytes against the layout to materialize a Value tree, then validate that Value against the compiled jsonschema validator. Decided in ADR-010.

pub fn validate_bytes(&self, buffer: &[u8]) -> Result<(), AlkTypeError>;

validate_bytes runs the existing machinery in sequence:

1. Materialize Value from bytes. A new internal helper (materialize_value, alongside SequentialReader::read_field_value in src/sequential_reader.rs) walks the buffer against the schema and the engine's Endian, producing a serde_json::Value tree. Composites are recursed into (Struct → object of field values; Array → array of element values; Union → dispatch then recurse; Record → object of key/value entries). The read phase reuses the existing data-access functions and returns AlkTypeError::Access (with field paths) on read failures.
2. Validate the Value. The materialized Value is passed to the existing self.validator.validate(&value), producing AlkTypeError::Validation on failure.

Mode dispatch:

• Packed mode — walks with a fresh SequentialReader (the engine is already a reader factory per ADR-007), materializing fields in declaration order.
• Aligned mode — uses the OffsetMap to read fields at their computed offsets, then materializes composites by recursing into the offset map's nested entries.

Both modes produce the same Value form; the validator is mode-agnostic (it operates on Value, not bytes — ADR-004).

When to use which entry point

Entry point	Schema form	Input form	When
validate_json(&Value)	Any (AlkType or plain JSON Schema)	Already-parsed serde_json::Value	Call's JSON payloads (OperationSpec.input_schema); TypeBox output; anything off serde_json::from_slice / from_str
validate_bytes(&[u8])	AlkType binary-layout schema	Raw &[u8] buffer	Channels' 8-byte chunk header; future binary call frames; SFTP packet buffers; metatensor index structs

validate_bytes requires the engine's schema to declare AlkType:* kinds — it materializes Value via the layout engine, which needs binary-layout semantics. A pure JSON Schema (call's input_schema, no AlkType kinds) compiled via AlkTypeEngine::compile would fail at the materialize step (no AlkType:Struct at the root). For pure JSON payloads, the consumer uses serde_json::from_slice then validate_json. See ADR-010 §"Not a binary-payload validator for JSON-only schemas".

What validate_bytes is not

• Not a new validation engine. It runs the existing jsonschema validator against the existing materialized Value. No new validator code, no parallel validation path (ADR-001).
• Not framing-aware. It validates the bytes of one schema instance. It does not strip length prefixes, parse [length: u32][payload] framing, or handle multiple frames in a buffer. That's the consumer's job. alktype validates what one schema describes; it does not parse the wire envelope around it.
• Not a Validator trait. Two methods on one struct, not a trait abstraction. See ADR-010 §"Not a Validator trait abstraction".

Relationship to Read/Write

Validation and data access are independent operations on the same data. The consumer can:

1. Validate the JSON representation of a buffer to ensure it conforms to the schema.
2. Read fields from the binary buffer at computed offsets.
3. Both — validate the JSON representation first, then read the binary buffer (defense in depth).

The engine does not couple validation and access. A consumer that trusts its data source can skip validation and go straight to read/write. A consumer that parses untrusted input can validate the JSON representation first, then access the binary buffer.

Design Decisions

Decision	ADR	Summary
Error handling and validation	ADR-004	AlkTypeError enum; load-time build, access-time check; field-path-carrying errors; jsonschema ValidationError wrapping
Generalized validation — validate_bytes	ADR-010	Single-call binary-buffer validation (materialize Value from bytes, then validate); two methods on one struct, not a trait
Purpose and scope	ADR-001	Why jsonschema not a custom engine

Open Questions

None specific to validation. The three alktype OQs (OQ-001, OQ-002, OQ-003) are about layout, platform support, and schema construction — not validation. OQ-003 is resolved by ADR-009; see builder.md.

References

• @alkdev/alknet: docs/research/alknet-typedef/findings.md §"Validation" — the POC's custom keyword validators for all 17 kinds
• ADR-004 — error handling and validation strategy
• schema-layer.md — the 19 AlkType kinds that the validators check
• data-access.md — read/write functions that operate on the same buffers