POC: /workspace/alktype-builder-poc/ (18/18 tests pass, findings in
docs/research/alktype-builder-poc/findings.md). Round 2 adds the SFTP
Packet validate_bytes tests (7 new: valid Init/Read/Write/Status,
short buffer, unknown discriminator, over-maxLength Bytes).
Open questions resolved (OQ-004 through OQ-008):
- OQ-004: Discriminator::Field name is String (already implemented;
docs updated to mark resolved)
- OQ-005: Both union discriminator kinds return the same shape:
{__discriminator, ...variant-fields}. Field-name path also had a
real offset bug (returned start, not end) - fixed.
- OQ-006: builder.md Example 3 now wraps the Union in a
Schema::struct_().field("payload", ...) and merges $defs via
Definitions::merge_into (matches the engine's AlkType:Struct-at-root
constraint and the SFTP wire shape)
- OQ-007: Bytes materialization is array-of-u8 (Value::Array of
Value::Number, one entry per byte 0..=255). BytesValidator accepts
both Value::String (validate_json) and Value::Array (validate_bytes).
maxLength = max byte count. Replaces the lossy from_utf8_lossy path
that corrupted non-UTF-8 bytes and broke maxLength semantics.
- OQ-008 (new): UnionValidator now dispatches to variant schemas via
sub-validators built at factory time. AlkTypeEngine::compile calls
schema::inline_union_variant_refs before build_validator to inline
$refs in union mapping entries (necessary because union_factory
receives the union node, but $defs live at the schema root).
Production-readiness fixes in src/ (no stubs/hedges in a published crate):
- materialize.rs: Record stub -> full count-prefixed key/value pair
implementation per schema-layer.md TRecord
- materialize.rs: root_of() was broken (returned the current node, not
the schema root) -> root schema threaded through every recursive call
so resolve_ref_or_inline can resolve $refs for nested composites
- builder.rs: LengthPrefixed encoding setter was a no-op when the
keyword was already in object form -> complete the branch (updates
the encoding entry in place for both LengthPrefixed and OffsetIndirect)
- builder.rs, engine.rs: POC-referencing comments cleaned up; the
round-trip test's or_else fallback (papering over write_field being
aligned-only) replaced with direct byte writes
Documentation:
- builder.md: Example 3 updated; Discriminator::Field spec shows String;
Open Questions section updated (OQ-004 resolved)
- validation.md: AlkType:Bytes and AlkType:Union validator descriptions
updated for array-of-u8 form and variant dispatch
- open-questions.md: OQ-004/005/006/007/008 marked resolved; new
Validation theme entries
- questions/004-008: individual OQ files updated with resolutions
- findings.md: round 2 results documented
Verification:
- cargo test: 369 -> 391 tests pass (22 new: 14 materialize, 5
inline_union_variant_refs, 3 validation/builder)
- cargo clippy --all-targets: clean
- POC: 11 -> 18 tests (7 new SFTP Packet tests); all pass
22 KiB
status, last_updated
| status | last_updated |
|---|---|
| draft | 2026-08-11 |
alktype — Builder API
The builder layer: a fluent Rust API for constructing alktype JSON
Schemas (both AlkType:*-bearing binary-layout schemas and plain
JSON-Schema-only operation payload schemas) at runtime, producing
serde_json::Value. Decided in ADR-009;
resolves OQ-003.
What
The builder module provides a single Schema builder type and a
Definitions helper for named $defs. The builder's .build() method
returns a serde_json::Value — the same form alktype already consumes
via AlkTypeEngine::compile (for AlkType:* schemas) and the same form
OperationSpec.input_schema / output_schema / error_schemas hold
(for plain JSON Schema, no AlkType:* kinds).
The builder covers:
- All 19
AlkType:*kinds (binary-layout schemas) — see schema-layer.md for the kinds. - All standard JSON Schema keywords needed for operation payload
schemas:
type,properties,required,items,enum,format,additionalProperties,minimum,maximum,minItems,maxItems,minLength,maxLength,$ref,$defs. - All ADR-003 schema annotations:
endian,align,encoding(length-prefixed / offset-indirect),maxLength-as-reservation, and TUniondiscriminator(byte-offset and field-name).
Why
alkcall (the merged alknet-call + alknet-channels extraction) is
alktype's first consumer and needs to build schemas at runtime from
Rust code, for two roles:
- Binary layout schemas (channels' 8-byte chunk header, future
binary call frames) —
AlkType:*schemas, fed toAlkTypeEngine::compile(packed mode, big-endian). - JSON payload schemas (call's
OperationSpec.input_schema/output_schema/error_schemas) — plain JSON Schema, noAlkType:*kinds, validated via the standardjsonschemavalidator.
A single builder serving both roles means alkcall imports one module
for schema construction. See ADR-009
for the decision rationale (why Value not a typed Schema enum, why
both AlkType and standard JSON Schema in one builder).
Architecture
Output type: serde_json::Value
The builder returns serde_json::Value. There is no typed Schema
enum. This matches ADR-001's "schemas are JSON" principle and avoids
duplicating the JSON form that AlkTypeEngine::compile,
jsonschema::Validator, and OperationSpec all consume.
Module placement
src/builder.rs, re-exported from the crate root. The builder is a
peer of schema.rs (which parses schemas) and engine.rs (which
compiles them). The builder constructs; it does not parse or compile.
// src/lib.rs (additions)
pub mod builder;
pub use builder::{Schema, Definitions};
Field order is load-bearing
serde_json with preserve_order is already a dependency (ADR-001).
The builder's Value output uses serde_json::Map (which preserves
insertion order under preserve_order), so field declaration order in
the builder is the field order in the binary layout. This is critical
for packed mode (ADR-002) where field order determines offsets.
Public API
Schema builder
Schema is the single entry point. Constructors for each AlkType kind
and each standard JSON Schema type; setters for annotations and
constraints; .build() produces Value.
AlkType kind constructors
One constructor per AlkTypeKind variant (see schema-layer.md
§"The 19 AlkType Kinds"):
impl Schema {
// Fixed-size integer kinds
pub fn int8() -> Self;
pub fn int16() -> Self;
pub fn int32() -> Self;
pub fn int64() -> Self;
pub fn uint8() -> Self;
pub fn uint16() -> Self;
pub fn uint32() -> Self;
pub fn uint64() -> Self;
// Float kinds
pub fn float32() -> Self;
pub fn float64() -> Self;
// Other fixed-size
pub fn boolean() -> Self;
pub fn enum_of(values: &[&str]) -> Self; // u32 index; values in declaration order
// Variable-length kinds
pub fn string() -> Self;
pub fn bytes() -> Self;
pub fn timestamp() -> Self;
// Composite kinds
pub fn struct_() -> Self; // fields added via .field()
pub fn union_(disc: Discriminator) -> Self; // variants via .mapping()
pub fn array_of(element: Schema) -> Self;
pub fn record_of(value: Schema) -> Self;
}
Each constructor sets the corresponding "AlkType:<Kind>": true key.
For example, Schema::uint32() produces {"AlkType:Uint32": true}.
enum_of sets both "AlkType:Enum": true and the standard
"enum" keyword with the provided values (declaration order is the
index order — see schema-layer.md §"TEnum binary
representation"):
Schema::enum_of(&["read", "write", "execute"])
// -> { "AlkType:Enum": true, "enum": ["read", "write", "execute"] }
array_of and record_of take the element/value schema as a
nested Schema:
Schema::array_of(Schema::uint32())
// -> { "AlkType:Array": true, "items": { "AlkType:Uint32": true } }
Schema::record_of(Schema::float32())
// -> { "AlkType:Record": true, "values": { "AlkType:Float32": true } }
Standard JSON Schema type constructors
For plain JSON Schema (no AlkType:* kinds) — call's
input_schema / output_schema / error_schemas:
impl Schema {
pub fn object() -> Self; // type: "object" — fields via .field()
pub fn array() -> Self; // type: "array" — items via .items()
pub fn string_() -> Self; // type: "string"
pub fn integer() -> Self; // type: "integer"
pub fn number() -> Self; // type: "number"
pub fn boolean_()-> Self; // type: "boolean"
pub fn null() -> Self; // type: "null"
pub fn any() -> Self; // {} (no type constraint)
}
The _ suffix disambiguates standard JSON Schema types from AlkType
kinds (string is the AlkType kind; string_ is the standard JSON
Schema type — the AlkType kind constructor sets "AlkType:String": true, the standard constructor sets "type": "string"). This is
deliberate: the two are distinct schema forms and the builder makes
the distinction visible at the call site.
Annotation setters
Annotation setters mirror ADR-003. Each setter is named after the
annotation it produces; calling the setter sets the corresponding JSON
key. Setters return Self for chaining.
impl Schema {
/// Schema-level endianness (ADR-003 §1). Default little.
pub fn endian(mut self, endian: Endian) -> Self;
/// Struct or field alignment (ADR-003 §2). Struct-level sets the
/// default; field-level overrides.
pub fn align(mut self, align: usize) -> Self;
/// Variable-length encoding strategy (ADR-003 §3).
/// Default length-prefixed.
pub fn encoding(mut self, encoding: VariableEncoding) -> Self;
/// maxLength — standard JSON Schema keyword. In aligned mode with
/// a variable-length type, reserves this many bytes (strategy 2).
/// In packed mode, validation constraint only.
pub fn max_length(mut self, max: usize) -> Self;
}
Endian and VariableEncoding are re-exported from schema.rs (no
new types — the builder uses the existing enums). The setters produce
the exact JSON shapes from ADR-003:
Schema::struct_()
.endian(Endian::Big)
.field("channel_id", Schema::uint32())
.field("length", Schema::uint32())
.build()
// -> {
// "AlkType:Struct": true,
// "endian": "big",
// "properties": {
// "channel_id": { "AlkType:Uint32": true },
// "length": { "AlkType:Uint32": true }
// }
// }
Composite builders
struct_(), union_(), array_of(), record_of() are the
composite constructors. struct_() and union_() need additional
setters to populate their children:
impl Schema {
/// Add a field to a struct (or object). Field order is load-bearing
/// for binary layouts (packed mode field order = byte order).
/// The field's schema is built from the passed `Schema`.
pub fn field(mut self, name: &str, field: Schema) -> Self;
/// Mark fields as required (standard JSON Schema `required` keyword).
/// Can be called multiple times; required names accumulate.
/// Field names must have been added via `.field()`.
pub fn required(mut self, names: &[&str]) -> Self;
/// Set the items schema for a standard `array` type.
pub fn items(mut self, item: Schema) -> Self;
/// Set the additionalProperties schema for a standard `object` type.
pub fn additional_properties(mut self, props: Schema) -> Self;
/// Add a variant to a union. `disc_value` is the stringified
/// discriminator value (mapping key). The variant schema is built
/// from the passed `Schema`.
pub fn mapping(mut self, disc_value: &str, variant: Schema) -> Self;
}
field sets properties[name] = field.build(). Repeated calls
append. Field order in the built Value is the call order (because
serde_json::Map preserves insertion order under preserve_order).
required sets the standard JSON Schema "required" array. The
builder does not check that the named fields exist (that's a
compile-time check at AlkTypeEngine::compile, per ADR-004's
load-time validation strategy — see ADR-009
§"What the builder is not"). Calling required multiple times
accumulates names:
Schema::object()
.field("path", Schema::string_())
.field("offset", Schema::integer().minimum(0))
.field("length", Schema::integer().minimum(0))
.required(["path"])
.required(["offset", "length"])
// -> {
// "type": "object",
// "properties": { "path": {...}, "offset": {...}, "length": {...} },
// "required": ["path", "offset", "length"]
// }
Constraint setters (standard JSON Schema)
For operation payload schemas (call's input_schema etc.):
impl Schema {
/// `minimum` (inclusive lower bound for numbers/integers).
pub fn minimum(mut self, min: f64) -> Self;
/// `maximum` (inclusive upper bound for numbers/integers).
pub fn maximum(mut self, max: f64) -> Self;
/// `minLength` (minimum string length).
pub fn min_length(mut self, min: usize) -> Self;
/// `minItems` (minimum array length).
pub fn min_items(mut self, min: usize) -> Self;
/// `maxItems` (maximum array length).
pub fn max_items(mut self, max: usize) -> Self;
/// `format` (e.g. "date-time", "uri", "email").
pub fn format(mut self, fmt: &str) -> Self;
/// `title` (human-readable description).
pub fn title(mut self, t: &str) -> Self;
/// `description` (human-readable description).
pub fn description(mut self, d: &str) -> Self;
}
These set the corresponding standard JSON Schema keywords. They apply
to both AlkType-kind schemas and standard JSON Schema type schemas
(e.g., Schema::string().max_length(4096) sets maxLength, which
serves as both a validation constraint and, in aligned mode, a
fixed-size reservation — ADR-003 §3).
.build()
impl Schema {
/// Produce the final `serde_json::Value`.
pub fn build(self) -> Value;
}
Consumes the builder and returns the assembled Value. The builder is
not Clone (to discourage partial builds); each Schema is consumed
once. To reuse a sub-schema, build it once and pass the Value to a
Schema::from_value constructor (below).
Schema::from_value — adopt an existing Value
impl Schema {
/// Adopt an existing JSON Schema `Value` as a `Schema`, for
/// composition with builder-constructed schemas. Does not validate
/// the schema; just wraps it.
pub fn from_value(value: Value) -> Self;
}
For consumers that have some schemas as JSON (e.g., loaded from a
TypeBox-produced file) and others built via the fluent API, and want
to compose them. from_value wraps the Value so it can be passed to
.field() / .items() / .mapping() like any other Schema.
Discriminator for union_()
union_() takes a Discriminator describing the union's dispatch
mechanism. This mirrors schema.rs::DiscriminatorKind but with a
builder-friendly shape (the kind enum is re-exported from schema.rs,
not duplicated):
pub enum Discriminator {
/// Byte-offset discriminator (ADR-003 §4 Kind A).
/// `offset` is the byte position; `disc_type` is the AlkType kind
/// of the discriminator (Uint8/Uint16/Uint32).
Byte {
offset: usize,
disc_type: AlkTypeKind, // restricted to Uint8/Uint16/Uint32
},
/// Field-name discriminator (ADR-003 §4 Kind B).
/// `name` is the field holding the discriminator value.
Field {
name: String,
},
}
Byte-offset example (channels-style protocol dispatch):
let packet = Schema::union_(Discriminator::Byte {
offset: 0,
disc_type: AlkTypeKind::Uint8,
})
.mapping("5", Schema::ref_def("Read"))
.mapping("6", Schema::ref_def("Write"))
.mapping("101", Schema::ref_def("Status"))
.build();
// -> {
// "AlkType:Union": true,
// "discriminator": { "kind": "byte", "offset": 0, "type": "AlkType:Uint8" },
// "mapping": { "5": {"$ref":"#/$defs/Read"}, "6": {...}, "101": {...} }
// }
Field-name example (typedef.ts pattern):
let event = Schema::union_(Discriminator::Field { name: "type" })
.mapping("read", Schema::ref_def("Read"))
.mapping("write", Schema::ref_def("Write"))
.build();
// -> {
// "AlkType:Union": true,
// "discriminator": { "kind": "field", "name": "type" },
// "mapping": { "read": {...}, "write": {...} }
// }
Definitions — named $defs for cross-reference
Definitions is a helper for building named $defs that schemas can
$ref by name. This is the ergonomics win for alkcall's
OperationSpec, where input/output/error schemas reference shared
definitions (e.g., FileNotFound, RateLimited).
pub struct Definitions { /* ... */ }
impl Definitions {
pub fn new() -> Self;
/// Define a named schema. Returns a `Schema` that produces
/// `{"$ref": "#/$defs/<name>"}` — the JSON Pointer form that
/// `jsonschema` and `AlkTypeEngine::compile` expect (after
/// `normalize_refs`, which the engine runs at compile time).
pub fn define(&mut self, name: &str, schema: Schema) -> Schema;
/// Like `define`, but the schema is an existing `Value` (adopted
/// via `Schema::from_value`).
pub fn define_value(&mut self, name: &str, value: Value) -> Schema;
/// Produce the `{"$defs": { ... }}` object to merge into a
/// top-level schema. Call once at the end.
pub fn build(self) -> Value;
}
Usage:
let mut defs = Definitions::new();
let file_not_found = defs.define("FileNotFound",
Schema::object()
.field("path", Schema::string_())
.field("errno", Schema::integer())
.required(["path", "errno"])
);
let rate_limited = defs.define("RateLimited",
Schema::object()
.field("retry_after_ms", Schema::integer().minimum(0))
.required(["retry_after_ms"])
);
let read_file_error = Schema::object()
.field("code", Schema::string_())
.field("details", Schema::any()) // one of the defined errors
.required(["code"])
.build();
// Merge $defs into the top-level schema that references them
let mut top = Schema::object()
.field("error", read_file_error)
.build();
top.as_object_mut().unwrap().insert("$defs".to_string(), defs.build());
define returns a Schema (the $ref to the definition), so it can
be passed directly to .field() / .mapping() / .items() without a
separate ref_def call:
let op = Schema::object()
.field("error", defs.define("FileNotFound", /* ... */))
.build();
Schema::ref_def — reference a definition by name
impl Schema {
/// Produce a `{"$ref": "#/$defs/<name>"}` schema. The definition
/// must exist in the `$defs` of the top-level schema at compile
/// time. The builder does not check this.
pub fn ref_def(name: &str) -> Self;
}
For cases where the Definitions::define return value isn't handy
(e.g., referencing a definition defined elsewhere). Produces the same
{"$ref": "#/$defs/<name>"} form.
Usage Examples
Example 1: channels' 8-byte chunk header (binary layout)
use alktype::{Schema, Endian};
let chunk_header = Schema::struct_()
.endian(Endian::Big)
.field("channel_id", Schema::uint32())
.field("length", Schema::uint32())
.build();
// -> {
// "AlkType:Struct": true,
// "endian": "big",
// "properties": {
// "channel_id": { "AlkType:Uint32": true },
// "length": { "AlkType:Uint32": true }
// }
// }
//
// Feed to AlkTypeEngine::compile(&mut chunk_header, LayoutMode::Packed)
// then validate incoming frames via engine.validate_bytes(&frame).
Example 2: call's OperationSpec input schema (JSON payload)
use alktype::Schema;
let read_file_input = Schema::object()
.field("path", Schema::string_().max_length(4096))
.field("offset", Schema::integer().minimum(0))
.field("length", Schema::integer().minimum(0))
.required(["path"])
.build();
// -> {
// "type": "object",
// "properties": {
// "path": { "type": "string", "maxLength": 4096 },
// "offset": { "type": "integer", "minimum": 0 },
// "length": { "type": "integer", "minimum": 0 }
// },
// "required": ["path"]
// }
//
// Stored in OperationSpec.input_schema; validated via the standard
// jsonschema validator (validate_json for parsed payloads, or via
// serde_json::from_slice then validate_json for wire frames).
Example 3: SFTP Packet union (binary layout, byte discriminator)
The SFTP wire shape is [type:u8][payload-struct] — a struct with a
union payload field. The engine requires AlkType:Struct at the top
level (OffsetMap::compute / SequentialReader::new both enforce
this; a Union is a field type within a struct, not a top-level
schema). The builder constructs the union wrapped in a struct, and
$defs are merged into the top-level schema so $refs resolve:
use alktype::{Definitions, Discriminator, AlkTypeKind, Schema};
let mut defs = Definitions::new();
defs.define("Init", Schema::struct_().field("version", Schema::uint32()));
defs.define("Open", Schema::struct_().field("path", Schema::string()).field("flags", Schema::uint32()));
defs.define("Read", Schema::struct_().field("handle", Schema::bytes()).field("offset", Schema::uint64()).field("len", Schema::uint32()));
defs.define("Write", Schema::struct_().field("handle", Schema::bytes()).field("offset", Schema::uint64()).field("data", Schema::bytes()));
defs.define("Status", Schema::struct_().field("code", Schema::uint32()).field("message", Schema::string()));
// A "Packet" is a struct with one field — the union. This mirrors
// SFTP's wire shape: [type:u8][payload-struct].
let mut packet = Schema::struct_()
.field(
"payload",
Schema::union_(Discriminator::Byte {
offset: 0,
disc_type: AlkTypeKind::Uint8,
})
.mapping("1", Schema::ref_def("Init"))
.mapping("3", Schema::ref_def("Open"))
.mapping("5", Schema::ref_def("Read"))
.mapping("6", Schema::ref_def("Write"))
.mapping("101", Schema::ref_def("Status")),
)
.build();
// Merge $defs into the top-level schema so $refs resolve at compile time.
defs.merge_into(&mut packet);
// Feed to AlkTypeEngine::compile(&mut packet, LayoutMode::Packed)
// then validate incoming frames via engine.validate_bytes(&frame).
Example 4: OperationSpec error schemas (named $defs)
use alktype::{Definitions, Schema};
let mut defs = Definitions::new();
let file_not_found = defs.define("FileNotFound",
Schema::object()
.field("path", Schema::string_())
.field("errno", Schema::integer())
.required(["path", "errno"])
);
let rate_limited = defs.define("RateLimited",
Schema::object()
.field("retry_after_ms", Schema::integer().minimum(0))
.required(["retry_after_ms"])
);
// An operation's error schemas reference these definitions
let op_errors = vec![
ErrorDefinition {
code: "FILE_NOT_FOUND".to_string(),
description: "File not found".to_string(),
schema: Schema::ref_def("FileNotFound").build(),
http_status: Some(404),
},
ErrorDefinition {
code: "RATE_LIMITED".to_string(),
description: "Rate limited".to_string(),
schema: Schema::ref_def("RateLimited").build(),
http_status: Some(429),
},
];
// $defs is built once and stored alongside the OperationSpec
Design Decisions
| Decision | ADR | Summary |
|---|---|---|
| Builder API for schema construction | ADR-009 | Fluent Rust API producing serde_json::Value; covers AlkType kinds + standard JSON Schema; resolves OQ-003 |
| Schema annotations | ADR-003 | The annotation shapes the builder's setters produce |
| Load-time validation strategy | ADR-004 | The builder does not pre-validate; compile-time is the validation point |
Open Questions
None specific to the builder. OQ-003 (the original "should we build a
builder API") is resolved by this spec / ADR-009. OQ-004
(Discriminator::Field name type — &str or String) is resolved:
String, for ownership simplicity (the builder consumes Self on
setters; &str would require a lifetime parameter on Discriminator
and transitively on Schema::union_). See
open-questions.md.
References
- ADR-009 — the decision this spec implements
- ADR-001 — scope boundaries this module extends; "schemas are JSON" principle
- ADR-003 — the annotation shapes the builder's setters produce
- schema-layer.md — the 19 AlkType kinds the builder's constructors produce
- validation.md — the validation layer that consumes
builder output (via
AlkTypeEngine::compile) @alkdev/alknet: docs/architecture/crates/call/operation-registry.md—OperationSpec(the alkcall consumer for the JSON-payload role)@alkdev/alknet: docs/architecture/crates/channels/channels-wire.md— the 8-byte chunk header (the alkcall consumer for the binary-layout role)