Draft builder API (ADR-009) and generalized validation validate_bytes (ADR-010) for v0.1.0

- ADR-009: fluent Rust builder producing serde_json::Value, covers
  AlkType kinds + standard JSON Schema; resolves OQ-003 (alkcall is the
  unblocking consumer)
- ADR-010: AlkTypeEngine::validate_bytes(&[u8]) as the single-call
  binary-buffer validation entry point; materialize Value from bytes,
  then validate; two methods on one struct, not a trait
- builder.md: full builder API spec (Schema, Definitions, Discriminator)
  with four usage examples (channels chunk header, call input schema,
  SFTP Packet union, OperationSpec error schemas)
- validation.md: new validate_bytes subsection + entry-point comparison
  table; AlkTypeEngine impl block updated; design decisions table updated
- overview.md: builder.md added to component pointers; 'Not a schema
  builder' scope boundary retired; ADR-009/010 added to decisions table;
  OQ-003 marked resolved; Consumers table adds alkcall as first consumer
- open-questions.md + questions/003: OQ-003 moved from deferred(scope)
  to resolved (ADR-009)
- README.md: builder doc + ADR-009/010 added; OQ-003 marked resolved;
  two new Key Design Principles (9, 10) for v0.1.0 additions

Doc-only change; 346 tests pass, clippy clean.
This commit is contained in:
2026-08-11 05:41:49 +00:00
parent 57d8ed25ba
commit 1a8a44ed0e
8 changed files with 1250 additions and 39 deletions

View File

@@ -1,6 +1,6 @@
---
status: draft
last_updated: 2026-07-22
last_updated: 2026-08-11
---
# alktype
@@ -18,7 +18,8 @@ format definition; the engine is generic.
| [schema-layer.md](schema-layer.md) | draft | The 19 `AlkType:*` kinds, jsonschema custom keyword integration, TypeBox interop, schema annotations |
| [layout-engine.md](layout-engine.md) | draft | Offset computation, the two layout modes (packed sequential vs aligned static), alignment, endianness, variable-length handling |
| [data-access.md](data-access.md) | draft | Read/write functions, TUnion dispatch, field paths, zero-copy access, length-prefix reading |
| [validation.md](validation.md) | draft | Custom keyword validators for all 19 `AlkType:*` kinds, `AlkTypeError`, load-time vs access-time validation, `AlkTypeEngine` |
| [validation.md](validation.md) | draft | Custom keyword validators for all 19 `AlkType:*` kinds, `AlkTypeError`, load-time vs access-time validation, `AlkTypeEngine`; `validate_bytes` for binary buffers (ADR-010) |
| [builder.md](builder.md) | draft | Fluent Rust API for constructing alktype JSON Schemas at runtime, producing `serde_json::Value`; covers AlkType kinds + standard JSON Schema (ADR-009) |
## Applicable ADRs
@@ -32,6 +33,8 @@ format definition; the engine is generic.
| [006](decisions/006-reject-non-final-inline-length-prefixed-in-aligned-mode.md) | Reject Non-Final Inline Length-Prefixed Variable Fields in Aligned Mode | Prevents silent data corruption (inline variable data clobbering subsequent fields) |
| [007](decisions/007-packed-mode-read-factory.md) | Packed-Mode Read API — Engine as SequentialReader Factory | `engine.sequential_reader()` returns an owned reader, not a reference |
| [008](decisions/008-reject-tunion-in-aligned-mode.md) | Reject TUnion in Aligned Mode for v1 | Unions are the protocol pattern; aligned-mode union semantics were broken |
| [009](decisions/009-builder-api.md) | Builder API for Schema Construction | Fluent Rust API producing `serde_json::Value`; covers AlkType kinds + standard JSON Schema; resolves OQ-003 |
| [010](decisions/010-generalized-validation-validate-bytes.md) | Generalized Validation — `validate_bytes` on `AlkTypeEngine` | Single-call binary-buffer validation; materialize `Value` from bytes, then validate; two methods on one struct, not a trait |
## Relevant Open Questions
@@ -39,7 +42,7 @@ format definition; the engine is generic.
|----|-------|--------|-----------|
| OQ-001 | Arrays of variable-length-element structs | deferred(scope) | Requires lazy walking logic; blocked on a concrete consumer that needs it |
| OQ-002 | `no_std` + `alloc` support | deferred(scope) | Target `std` for v1; blocked on an embedded use case |
| OQ-003 | Builder API for schema construction | deferred(scope) | Schemas are authored in TypeBox or hand-written JSON for v1; blocked on a concrete need |
| OQ-003 | Builder API for schema construction | resolved (ADR-009) | Resolved in v0.1.0; alkcall is the concrete consumer; see [builder.md](builder.md) |
## Key Design Principles
@@ -95,6 +98,21 @@ format definition; the engine is generic.
with a known schema, use alktype. See [overview.md](overview.md) and
[ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md).
9. **Schemas can be built at runtime from Rust (v0.1.0).** A fluent
builder API produces `serde_json::Value` for both AlkType-kind
schemas and standard JSON Schema, covering alkcall's two roles
(binary layout + JSON payloads) from one module. The builder is
additive — consumers with static schemas continue to load JSON.
See [builder.md](builder.md) and [ADR-009](decisions/009-builder-api.md).
10. **Two validation entry points, one engine (v0.1.0).**
`validate_json(&Value)` for already-parsed JSON (call's payloads);
`validate_bytes(&[u8])` for binary buffers (channels' chunk header).
Same underlying `jsonschema` validator; the bytes path materializes
a `Value` tree via the layout engine, then validates. See
[validation.md](validation.md) and
[ADR-010](decisions/010-generalized-validation-validate-bytes.md).
## References
- `@alkdev/alknet: docs/research/alknet-typedef/findings.md` — POC results

View File

@@ -0,0 +1,631 @@
---
status: draft
last_updated: 2026-08-11
---
# alktype — Builder API
The builder layer: a fluent Rust API for constructing alktype JSON
Schemas (both `AlkType:*`-bearing binary-layout schemas and plain
JSON-Schema-only operation payload schemas) at runtime, producing
`serde_json::Value`. Decided in [ADR-009](decisions/009-builder-api.md);
resolves [OQ-003](questions/003-builder-api-for-schema-construction.md).
## What
The `builder` module provides a single `Schema` builder type and a
`Definitions` helper for named `$defs`. The builder's `.build()` method
returns a `serde_json::Value` — the same form alktype already consumes
via `AlkTypeEngine::compile` (for `AlkType:*` schemas) and the same form
`OperationSpec.input_schema` / `output_schema` / `error_schemas` hold
(for plain JSON Schema, no `AlkType:*` kinds).
The builder covers:
- All 19 `AlkType:*` kinds (binary-layout schemas) — see
[schema-layer.md](schema-layer.md) for the kinds.
- All standard JSON Schema keywords needed for operation payload
schemas: `type`, `properties`, `required`, `items`, `enum`, `format`,
`additionalProperties`, `minimum`, `maximum`, `minItems`,
`maxItems`, `minLength`, `maxLength`, `$ref`, `$defs`.
- All ADR-003 schema annotations: `endian`, `align`, `encoding`
(length-prefixed / offset-indirect), `maxLength`-as-reservation, and
TUnion `discriminator` (byte-offset and field-name).
## Why
alkcall (the merged `alknet-call` + `alknet-channels` extraction) is
alktype's first consumer and needs to build schemas at runtime from
Rust code, for two roles:
1. **Binary layout schemas** (channels' 8-byte chunk header, future
binary call frames) — `AlkType:*` schemas, fed to
`AlkTypeEngine::compile` (packed mode, big-endian).
2. **JSON payload schemas** (call's `OperationSpec.input_schema` /
`output_schema` / `error_schemas`) — plain JSON Schema, no
`AlkType:*` kinds, validated via the standard `jsonschema` validator.
A single builder serving both roles means alkcall imports one module
for schema construction. See [ADR-009](decisions/009-builder-api.md)
for the decision rationale (why `Value` not a typed `Schema` enum, why
both AlkType and standard JSON Schema in one builder).
## Architecture
### Output type: `serde_json::Value`
The builder returns `serde_json::Value`. There is no typed `Schema`
enum. This matches ADR-001's "schemas are JSON" principle and avoids
duplicating the JSON form that `AlkTypeEngine::compile`,
`jsonschema::Validator`, and `OperationSpec` all consume.
### Module placement
`src/builder.rs`, re-exported from the crate root. The builder is a
peer of `schema.rs` (which parses schemas) and `engine.rs` (which
compiles them). The builder constructs; it does not parse or compile.
```rust
// src/lib.rs (additions)
pub mod builder;
pub use builder::{Schema, Definitions};
```
### Field order is load-bearing
`serde_json` with `preserve_order` is already a dependency (ADR-001).
The builder's `Value` output uses `serde_json::Map` (which preserves
insertion order under `preserve_order`), so field declaration order in
the builder is the field order in the binary layout. This is critical
for packed mode (ADR-002) where field order determines offsets.
## Public API
### `Schema` builder
`Schema` is the single entry point. Constructors for each AlkType kind
and each standard JSON Schema type; setters for annotations and
constraints; `.build()` produces `Value`.
#### AlkType kind constructors
One constructor per `AlkTypeKind` variant (see [schema-layer.md](schema-layer.md)
§"The 19 AlkType Kinds"):
```rust
impl Schema {
// Fixed-size integer kinds
pub fn int8() -> Self;
pub fn int16() -> Self;
pub fn int32() -> Self;
pub fn int64() -> Self;
pub fn uint8() -> Self;
pub fn uint16() -> Self;
pub fn uint32() -> Self;
pub fn uint64() -> Self;
// Float kinds
pub fn float32() -> Self;
pub fn float64() -> Self;
// Other fixed-size
pub fn boolean() -> Self;
pub fn enum_of(values: &[&str]) -> Self; // u32 index; values in declaration order
// Variable-length kinds
pub fn string() -> Self;
pub fn bytes() -> Self;
pub fn timestamp() -> Self;
// Composite kinds
pub fn struct_() -> Self; // fields added via .field()
pub fn union_(disc: Discriminator) -> Self; // variants via .mapping()
pub fn array_of(element: Schema) -> Self;
pub fn record_of(value: Schema) -> Self;
}
```
Each constructor sets the corresponding `"AlkType:<Kind>": true` key.
For example, `Schema::uint32()` produces `{"AlkType:Uint32": true}`.
**`enum_of`** sets both `"AlkType:Enum": true` and the standard
`"enum"` keyword with the provided values (declaration order is the
index order — see [schema-layer.md](schema-layer.md) §"TEnum binary
representation"):
```rust
Schema::enum_of(&["read", "write", "execute"])
// -> { "AlkType:Enum": true, "enum": ["read", "write", "execute"] }
```
**`array_of`** and **`record_of`** take the element/value schema as a
nested `Schema`:
```rust
Schema::array_of(Schema::uint32())
// -> { "AlkType:Array": true, "items": { "AlkType:Uint32": true } }
Schema::record_of(Schema::float32())
// -> { "AlkType:Record": true, "values": { "AlkType:Float32": true } }
```
#### Standard JSON Schema type constructors
For plain JSON Schema (no `AlkType:*` kinds) — call's
`input_schema` / `output_schema` / `error_schemas`:
```rust
impl Schema {
pub fn object() -> Self; // type: "object" — fields via .field()
pub fn array() -> Self; // type: "array" — items via .items()
pub fn string_() -> Self; // type: "string"
pub fn integer() -> Self; // type: "integer"
pub fn number() -> Self; // type: "number"
pub fn boolean_()-> Self; // type: "boolean"
pub fn null() -> Self; // type: "null"
pub fn any() -> Self; // {} (no type constraint)
}
```
The `_` suffix disambiguates standard JSON Schema types from AlkType
kinds (`string` is the AlkType kind; `string_` is the standard JSON
Schema type — the AlkType kind constructor sets `"AlkType:String":
true`, the standard constructor sets `"type": "string"`). This is
deliberate: the two are distinct schema forms and the builder makes
the distinction visible at the call site.
#### Annotation setters
Annotation setters mirror ADR-003. Each setter is named after the
annotation it produces; calling the setter sets the corresponding JSON
key. Setters return `Self` for chaining.
```rust
impl Schema {
/// Schema-level endianness (ADR-003 §1). Default little.
pub fn endian(mut self, endian: Endian) -> Self;
/// Struct or field alignment (ADR-003 §2). Struct-level sets the
/// default; field-level overrides.
pub fn align(mut self, align: usize) -> Self;
/// Variable-length encoding strategy (ADR-003 §3).
/// Default length-prefixed.
pub fn encoding(mut self, encoding: VariableEncoding) -> Self;
/// maxLength — standard JSON Schema keyword. In aligned mode with
/// a variable-length type, reserves this many bytes (strategy 2).
/// In packed mode, validation constraint only.
pub fn max_length(mut self, max: usize) -> Self;
}
```
`Endian` and `VariableEncoding` are re-exported from `schema.rs` (no
new types — the builder uses the existing enums). The setters produce
the exact JSON shapes from ADR-003:
```rust
Schema::struct_()
.endian(Endian::Big)
.field("channel_id", Schema::uint32())
.field("length", Schema::uint32())
.build()
// -> {
// "AlkType:Struct": true,
// "endian": "big",
// "properties": {
// "channel_id": { "AlkType:Uint32": true },
// "length": { "AlkType:Uint32": true }
// }
// }
```
#### Composite builders
`struct_()`, `union_()`, `array_of()`, `record_of()` are the
composite constructors. `struct_()` and `union_()` need additional
setters to populate their children:
```rust
impl Schema {
/// Add a field to a struct (or object). Field order is load-bearing
/// for binary layouts (packed mode field order = byte order).
/// The field's schema is built from the passed `Schema`.
pub fn field(mut self, name: &str, field: Schema) -> Self;
/// Mark fields as required (standard JSON Schema `required` keyword).
/// Can be called multiple times; required names accumulate.
/// Field names must have been added via `.field()`.
pub fn required(mut self, names: &[&str]) -> Self;
/// Set the items schema for a standard `array` type.
pub fn items(mut self, item: Schema) -> Self;
/// Set the additionalProperties schema for a standard `object` type.
pub fn additional_properties(mut self, props: Schema) -> Self;
/// Add a variant to a union. `disc_value` is the stringified
/// discriminator value (mapping key). The variant schema is built
/// from the passed `Schema`.
pub fn mapping(mut self, disc_value: &str, variant: Schema) -> Self;
}
```
**`field`** sets `properties[name] = field.build()`. Repeated calls
append. Field order in the built `Value` is the call order (because
`serde_json::Map` preserves insertion order under `preserve_order`).
**`required`** sets the standard JSON Schema `"required"` array. The
builder does not check that the named fields exist (that's a
compile-time check at `AlkTypeEngine::compile`, per ADR-004's
load-time validation strategy — see [ADR-009](decisions/009-builder-api.md)
§"What the builder is not"). Calling `required` multiple times
accumulates names:
```rust
Schema::object()
.field("path", Schema::string_())
.field("offset", Schema::integer().minimum(0))
.field("length", Schema::integer().minimum(0))
.required(["path"])
.required(["offset", "length"])
// -> {
// "type": "object",
// "properties": { "path": {...}, "offset": {...}, "length": {...} },
// "required": ["path", "offset", "length"]
// }
```
#### Constraint setters (standard JSON Schema)
For operation payload schemas (call's `input_schema` etc.):
```rust
impl Schema {
/// `minimum` (inclusive lower bound for numbers/integers).
pub fn minimum(mut self, min: f64) -> Self;
/// `maximum` (inclusive upper bound for numbers/integers).
pub fn maximum(mut self, max: f64) -> Self;
/// `minLength` (minimum string length).
pub fn min_length(mut self, min: usize) -> Self;
/// `minItems` (minimum array length).
pub fn min_items(mut self, min: usize) -> Self;
/// `maxItems` (maximum array length).
pub fn max_items(mut self, max: usize) -> Self;
/// `format` (e.g. "date-time", "uri", "email").
pub fn format(mut self, fmt: &str) -> Self;
/// `title` (human-readable description).
pub fn title(mut self, t: &str) -> Self;
/// `description` (human-readable description).
pub fn description(mut self, d: &str) -> Self;
}
```
These set the corresponding standard JSON Schema keywords. They apply
to both AlkType-kind schemas and standard JSON Schema type schemas
(e.g., `Schema::string().max_length(4096)` sets `maxLength`, which
serves as both a validation constraint and, in aligned mode, a
fixed-size reservation — ADR-003 §3).
#### `.build()`
```rust
impl Schema {
/// Produce the final `serde_json::Value`.
pub fn build(self) -> Value;
}
```
Consumes the builder and returns the assembled `Value`. The builder is
not `Clone` (to discourage partial builds); each `Schema` is consumed
once. To reuse a sub-schema, build it once and pass the `Value` to a
`Schema::from_value` constructor (below).
#### `Schema::from_value` — adopt an existing Value
```rust
impl Schema {
/// Adopt an existing JSON Schema `Value` as a `Schema`, for
/// composition with builder-constructed schemas. Does not validate
/// the schema; just wraps it.
pub fn from_value(value: Value) -> Self;
}
```
For consumers that have some schemas as JSON (e.g., loaded from a
TypeBox-produced file) and others built via the fluent API, and want
to compose them. `from_value` wraps the `Value` so it can be passed to
`.field()` / `.items()` / `.mapping()` like any other `Schema`.
### `Discriminator` for `union_()`
`union_()` takes a `Discriminator` describing the union's dispatch
mechanism. This mirrors `schema.rs::DiscriminatorKind` but with a
builder-friendly shape (the kind enum is re-exported from `schema.rs`,
not duplicated):
```rust
pub enum Discriminator {
/// Byte-offset discriminator (ADR-003 §4 Kind A).
/// `offset` is the byte position; `disc_type` is the AlkType kind
/// of the discriminator (Uint8/Uint16/Uint32).
Byte {
offset: usize,
disc_type: AlkTypeKind, // restricted to Uint8/Uint16/Uint32
},
/// Field-name discriminator (ADR-003 §4 Kind B).
/// `name` is the field holding the discriminator value.
Field {
name: &str, // or String — see "Open question" below
},
}
```
**Byte-offset example** (channels-style protocol dispatch):
```rust
let packet = Schema::union_(Discriminator::Byte {
offset: 0,
disc_type: AlkTypeKind::Uint8,
})
.mapping("5", Schema::ref_def("Read"))
.mapping("6", Schema::ref_def("Write"))
.mapping("101", Schema::ref_def("Status"))
.build();
// -> {
// "AlkType:Union": true,
// "discriminator": { "kind": "byte", "offset": 0, "type": "AlkType:Uint8" },
// "mapping": { "5": {"$ref":"#/$defs/Read"}, "6": {...}, "101": {...} }
// }
```
**Field-name example** (typedef.ts pattern):
```rust
let event = Schema::union_(Discriminator::Field { name: "type" })
.mapping("read", Schema::ref_def("Read"))
.mapping("write", Schema::ref_def("Write"))
.build();
// -> {
// "AlkType:Union": true,
// "discriminator": { "kind": "field", "name": "type" },
// "mapping": { "read": {...}, "write": {...} }
// }
```
### `Definitions` — named `$defs` for cross-reference
`Definitions` is a helper for building named `$defs` that schemas can
`$ref` by name. This is the ergonomics win for alkcall's
`OperationSpec`, where input/output/error schemas reference shared
definitions (e.g., `FileNotFound`, `RateLimited`).
```rust
pub struct Definitions { /* ... */ }
impl Definitions {
pub fn new() -> Self;
/// Define a named schema. Returns a `Schema` that produces
/// `{"$ref": "#/$defs/<name>"}` — the JSON Pointer form that
/// `jsonschema` and `AlkTypeEngine::compile` expect (after
/// `normalize_refs`, which the engine runs at compile time).
pub fn define(&mut self, name: &str, schema: Schema) -> Schema;
/// Like `define`, but the schema is an existing `Value` (adopted
/// via `Schema::from_value`).
pub fn define_value(&mut self, name: &str, value: Value) -> Schema;
/// Produce the `{"$defs": { ... }}` object to merge into a
/// top-level schema. Call once at the end.
pub fn build(self) -> Value;
}
```
**Usage:**
```rust
let mut defs = Definitions::new();
let file_not_found = defs.define("FileNotFound",
Schema::object()
.field("path", Schema::string_())
.field("errno", Schema::integer())
.required(["path", "errno"])
);
let rate_limited = defs.define("RateLimited",
Schema::object()
.field("retry_after_ms", Schema::integer().minimum(0))
.required(["retry_after_ms"])
);
let read_file_error = Schema::object()
.field("code", Schema::string_())
.field("details", Schema::any()) // one of the defined errors
.required(["code"])
.build();
// Merge $defs into the top-level schema that references them
let mut top = Schema::object()
.field("error", read_file_error)
.build();
top.as_object_mut().unwrap().insert("$defs".to_string(), defs.build());
```
`define` returns a `Schema` (the `$ref` to the definition), so it can
be passed directly to `.field()` / `.mapping()` / `.items()` without a
separate `ref_def` call:
```rust
let op = Schema::object()
.field("error", defs.define("FileNotFound", /* ... */))
.build();
```
#### `Schema::ref_def` — reference a definition by name
```rust
impl Schema {
/// Produce a `{"$ref": "#/$defs/<name>"}` schema. The definition
/// must exist in the `$defs` of the top-level schema at compile
/// time. The builder does not check this.
pub fn ref_def(name: &str) -> Self;
}
```
For cases where the `Definitions::define` return value isn't handy
(e.g., referencing a definition defined elsewhere). Produces the same
`{"$ref": "#/$defs/<name>"}` form.
## Usage Examples
### Example 1: channels' 8-byte chunk header (binary layout)
```rust
use alktype::{Schema, Endian};
let chunk_header = Schema::struct_()
.endian(Endian::Big)
.field("channel_id", Schema::uint32())
.field("length", Schema::uint32())
.build();
// -> {
// "AlkType:Struct": true,
// "endian": "big",
// "properties": {
// "channel_id": { "AlkType:Uint32": true },
// "length": { "AlkType:Uint32": true }
// }
// }
//
// Feed to AlkTypeEngine::compile(&mut chunk_header, LayoutMode::Packed)
// then validate incoming frames via engine.validate_bytes(&frame).
```
### Example 2: call's `OperationSpec` input schema (JSON payload)
```rust
use alktype::Schema;
let read_file_input = Schema::object()
.field("path", Schema::string_().max_length(4096))
.field("offset", Schema::integer().minimum(0))
.field("length", Schema::integer().minimum(0))
.required(["path"])
.build();
// -> {
// "type": "object",
// "properties": {
// "path": { "type": "string", "maxLength": 4096 },
// "offset": { "type": "integer", "minimum": 0 },
// "length": { "type": "integer", "minimum": 0 }
// },
// "required": ["path"]
// }
//
// Stored in OperationSpec.input_schema; validated via the standard
// jsonschema validator (validate_json for parsed payloads, or via
// serde_json::from_slice then validate_json for wire frames).
```
### Example 3: SFTP `Packet` union (binary layout, byte discriminator)
```rust
use alktype::{Schema, Discriminator, AlkTypeKind};
let mut defs = Definitions::new();
defs.define("Init", Schema::struct_().field("version", Schema::uint32()));
defs.define("Open", Schema::struct_().field("path", Schema::string()).field("flags", Schema::uint32()));
defs.define("Read", Schema::struct_().field("handle", Schema::bytes()).field("offset", Schema::uint64()).field("len", Schema::uint32()));
defs.define("Write", Schema::struct_().field("handle", Schema::bytes()).field("offset", Schema::uint64()).field("data", Schema::bytes()));
defs.define("Status", Schema::struct_().field("code", Schema::uint32()).field("message", Schema::string()));
let packet = Schema::union_(Discriminator::Byte {
offset: 0,
disc_type: AlkTypeKind::Uint8,
})
.mapping("1", Schema::ref_def("Init"))
.mapping("3", Schema::ref_def("Open"))
.mapping("5", Schema::ref_def("Read"))
.mapping("6", Schema::ref_def("Write"))
.mapping("101", Schema::ref_def("Status"))
.build();
```
### Example 4: OperationSpec error schemas (named `$defs`)
```rust
use alktype::{Definitions, Schema};
let mut defs = Definitions::new();
let file_not_found = defs.define("FileNotFound",
Schema::object()
.field("path", Schema::string_())
.field("errno", Schema::integer())
.required(["path", "errno"])
);
let rate_limited = defs.define("RateLimited",
Schema::object()
.field("retry_after_ms", Schema::integer().minimum(0))
.required(["retry_after_ms"])
);
// An operation's error schemas reference these definitions
let op_errors = vec![
ErrorDefinition {
code: "FILE_NOT_FOUND".to_string(),
description: "File not found".to_string(),
schema: Schema::ref_def("FileNotFound").build(),
http_status: Some(404),
},
ErrorDefinition {
code: "RATE_LIMITED".to_string(),
description: "Rate limited".to_string(),
schema: Schema::ref_def("RateLimited").build(),
http_status: Some(429),
},
];
// $defs is built once and stored alongside the OperationSpec
```
## Design Decisions
| Decision | ADR | Summary |
|----------|-----|---------|
| Builder API for schema construction | [ADR-009](decisions/009-builder-api.md) | Fluent Rust API producing `serde_json::Value`; covers AlkType kinds + standard JSON Schema; resolves OQ-003 |
| Schema annotations | [ADR-003](decisions/003-schema-annotations.md) | The annotation shapes the builder's setters produce |
| Load-time validation strategy | [ADR-004](decisions/004-error-handling-validation-strategy.md) | The builder does not pre-validate; compile-time is the validation point |
## Open Questions
- **OQ-004** (open): `Discriminator::Field` name type — `&str` or
`String`? The builder consumes `Self` on each setter, so the
discriminator's `name` needs to outlive the `Schema` it's embedded
in. `&str` borrows; `String` owns. Resolve during implementation
(likely `String` for ownership simplicity, since the builder owns
its content until `.build()`).
## References
- [ADR-009](decisions/009-builder-api.md) — the decision this spec implements
- [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md) —
scope boundaries this module extends; "schemas are JSON" principle
- [ADR-003](decisions/003-schema-annotations.md) — the annotation
shapes the builder's setters produce
- [schema-layer.md](schema-layer.md) — the 19 AlkType kinds the
builder's constructors produce
- [validation.md](validation.md) — the validation layer that consumes
builder output (via `AlkTypeEngine::compile`)
- `@alkdev/alknet: docs/architecture/crates/call/operation-registry.md`
`OperationSpec` (the alkcall consumer for the JSON-payload role)
- `@alkdev/alknet: docs/architecture/crates/channels/channels-wire.md`
— the 8-byte chunk header (the alkcall consumer for the
binary-layout role)

View File

@@ -0,0 +1,205 @@
# ADR-009: Builder API for Schema Construction
## Status
Accepted
## Context
alktype v0.1.0 resolves OQ-003 ("Builder API for schema construction",
deferred since ADR-001's scope boundaries). The deferral's blocking
condition was "a concrete need for programmatic schema construction in
Rust." That condition has arrived: **alkcall** — the crate formed by
extracting and merging `@alkdev/alknet`'s `alknet-call` (call protocol)
and `alknet-channels` (channel multiplexing) — is alktype's first
consumer, and it builds schemas at runtime from Rust code rather than
loading pre-authored JSON.
alkcall needs alktype for two distinct schema roles, and the builder
must serve both:
1. **Binary layout schemas** for channels' 8-byte chunk header
(`{ channel_id: u32 BE, length: u32 BE }`) and any future binary
call frames. These use `AlkType:*` custom keywords and feed
`AlkTypeEngine::compile` (packed mode, big-endian).
2. **JSON payload schemas** for `OperationSpec.input_schema` /
`output_schema` / `error_schemas` (ADR-023 in alknet). These are
plain JSON Schema — **no `AlkType:*` keywords** — and validate via
the standard `jsonschema` validator (the same one alktype already
builds through `build_validator`).
Today both roles are built with `serde_json::json!({...})` literals or
TypeBox output, which is correct for static schemas authored in JS but
ergonomically poor and error-prone for schemas assembled at runtime in
Rust. Operation specs in alknet-call today are constructed via
`serde_json::Value` literals; named `$defs` are awkward; cross-references
between input/output/error schemas require manual JSON Pointer
construction.
The "schema is the format" principle (ADR-001) is unchanged: the
builder produces `serde_json::Value`, the same form alktype already
consumes. The builder is a **construction** concern, not a new schema
representation.
## Decision
**alktype v0.1.0 ships a fluent Rust builder API in a new
`builder` module that produces `serde_json::Value`.** The builder covers
both the 19 `AlkType:*` kinds and the standard JSON Schema keywords
(`type`, `properties`, `required`, `items`, `enum`, `format`,
`additionalProperties`, `$ref`, `$defs`, etc.) so it can serve alkcall's
two roles from one module.
**Output type: `serde_json::Value`.** No parallel typed `Schema` enum.
The builder is a thin construction layer over the JSON form alktype
already consumes; introducing a second representation would duplicate
the schema's structure and create a sync hazard with the JSON wire form
that TypeBox, `jsonschema`, and `OperationSpec` all use. This matches
the principle from ADR-001: schemas are JSON; the engine consumes JSON;
the builder is a convenient way to *write* JSON from Rust.
**Scope: both AlkType kinds and standard JSON Schema.** A single
`Schema` builder type covers both. Pure JSON Schema (call's
`input_schema`) is just an `AlkType:*`-free schema; the builder produces
the same `Value` either way. Splitting "AlkType builder" and "JSON
Schema builder" into two types would force alkcall to use two builders
for the two roles and would invent a boundary that doesn't exist in the
underlying format. One builder, one `Value` output.
**Module placement: `src/builder.rs`, re-exported from the crate root.**
Sits alongside `schema.rs` (which parses schemas) and `engine.rs` (which
compiles them). The builder is a peer of `schema.rs`'s parsing API, not
inside the engine — it constructs schemas, it doesn't consume them.
### Builder surface (summary)
The detailed API is in [../builder.md](../builder.md). Summary:
```rust
// Channels' 8-byte chunk header — AlkType schema, big-endian, packed mode
let chunk_header = Schema::struct_()
.endian(Endian::Big)
.field("channel_id", Schema::uint32())
.field("length", Schema::uint32())
.build();
// call's OperationSpec input schema — plain JSON Schema, no AlkType kinds
let read_file_input = Schema::object()
.field("path", Schema::string().max_length(4096))
.field("offset", Schema::integer().minimum(0))
.field("length", Schema::integer().minimum(0))
.required(["path"])
.build();
// Named $defs for cross-reference (e.g. OperationSpec with shared error schemas)
let mut defs = Definitions::new();
let file_not_found = defs.define("FileNotFound",
Schema::object()
.field("path", Schema::string())
.field("errno", Schema::integer())
.build());
let read_file = Schema::ref_def("FileNotFound"); // "#/$defs/FileNotFound"
```
### What the builder is not
- **Not a typed schema representation.** The builder returns `Value`;
there is no `Schema` enum that consumers match on. The builder
*constructs*; it does not *represent*.
- **Not a validator of schema well-formedness.** The builder produces
JSON that *should* be a valid JSON Schema, but it does not itself
validate the schema against the JSON Schema meta-schema. That check
happens at `AlkTypeEngine::compile` (which calls `build_validator`,
which delegates to `jsonschema` — see ADR-004). A consumer that wants
to validate a pure-JSON-Schema builder output (no AlkType kinds)
against the meta-schema can do so via `jsonschema` directly; alktype
does not add a separate path for that in v0.1.0.
- **Not a replacement for TypeBox or hand-authored JSON.** Consumers
with static schemas continue to load JSON. The builder is for
programmatic construction at runtime.
- **Not a code generator.** typebox-rs's `codegen/` module (ADR-001
scope boundary) stays out of scope. The builder produces runtime
`Value`, not Rust source.
## Consequences
### Positive
- **alkcall has one library for both schema roles.** Binary chunk
headers and JSON operation payloads built via the same fluent API,
imported from the same crate.
- **Type-safe construction from Rust.** Field names, kinds, and
annotations are checked at compile time by Rust's type system as far
as the builder's signatures allow. Typos in `"AlkType:Uint32"` become
`Schema::uint32()` call-site errors.
- **Named definitions are first-class.** `Definitions` makes
`$ref`/`$defs` ergonomic — alkcall can build an `OperationSpec` with
named error schemas (`FileNotFound`, `RateLimited`) and reference
them by name from input/output schemas without hand-writing JSON
Pointers.
- **Purely additive.** No existing API changes. Consumers that load
JSON continue to load JSON. The builder is a new module, a new set of
re-exports, and a resolved OQ.
- **Resolves OQ-003.** The blocking condition (concrete need for
programmatic schema construction in Rust) is met by alkcall.
### Negative
- **New public API surface to maintain.** The `builder` module's
constructors and setters become part of the public API. Renaming or
removing a constructor is a breaking change. The surface is
minimized by mirroring the existing `AlkTypeKind` enum and ADR-003
annotations rather than inventing new vocabulary.
- **No compile-time guarantee that builder output is a valid JSON
Schema.** A consumer can call `Schema::struct_().field("x",
Schema::integer()).build()` and get a `Value` that fails
`AlkTypeEngine::compile` for some other reason (e.g., a
variable-length field in a position the layout engine rejects —
ADR-006). The builder does not pre-validate; compile-time is the
validation point. This is consistent with ADR-004's load-time
validation strategy.
- **Two ways to build schemas.** Consumers can now build a schema via
the fluent API or via `serde_json::json!()`. Both produce `Value`.
The builder is preferred for runtime construction; `json!` remains
fine for static test fixtures. This is a two-way door: the builder
can be added without changing the JSON path, and either can be
removed without affecting the other.
## Scope Boundaries (What This Is Not)
- **No typed `Schema` enum.** A typed Rust representation of schemas
(the "Schema struct → Value" option we considered) is explicitly
rejected. It would duplicate the JSON form, create a sync hazard with
TypeBox/`jsonschema`/`OperationSpec` which all use `Value`, and
require a bidirectional conversion layer. The builder returns
`Value`.
- **No schema-to-Rust-types codegen.** Code generation is out of scope
(ADR-001). The builder produces runtime `Value`, not source.
- **No schema evolution / Value system.** TypeBox's `Value.Diff`,
`Value.Migrate`, `Value.Convert` remain out of scope (ADR-001). The
builder constructs; it does not diff, migrate, or convert.
- **No meta-schema validation in the builder.** Validating builder
output against the JSON Schema meta-schema is the consumer's job
(via `AlkTypeEngine::compile` for AlkType schemas, or via
`jsonschema` directly for pure JSON Schema). Adding it to the
builder would either duplicate `jsonschema`'s work or couple the
builder to `jsonschema` at construction time. Neither is justified
for v0.1.0.
## References
- [OQ-003](../questions/003-builder-api-for-schema-construction.md) —
the resolved open question this ADR closes
- [builder.md](../builder.md) — the spec doc with the full builder API
- [ADR-001](001-alktype-purpose-scope-jsonschema-engine.md) — scope
boundaries this ADR extends; the "not a schema builder" boundary is
retired by this ADR
- [ADR-003](003-schema-annotations.md) — the annotation shapes the
builder's setters produce
- [ADR-004](004-error-handling-validation-strategy.md) — load-time
validation strategy; the builder does not pre-validate
- `@alkdev/alknet: docs/architecture/crates/call/operation-registry.md`
`OperationSpec` (the alkcall consumer that needs this builder)
- `@alkdev/alknet: docs/architecture/crates/channels/channels-wire.md`
— the 8-byte chunk header (the alkcall consumer that needs the
binary-layout role)

View File

@@ -0,0 +1,255 @@
# ADR-010: Generalized Validation — `validate_bytes` on `AlkTypeEngine`
## Status
Accepted
## Context
alktype v0.1.0 adds a generalized validation API so that **alkcall**
the merged `alknet-call` + `alknet-channels` extraction — can hold a
single library for validating both its binary wire frames and its JSON
payloads.
alktype's validation today (ADR-004, validation.md §"What validation
validates") is JSON-only by design:
- `AlkTypeEngine::validate_json(&Value)` and `is_valid_json(&Value)`
operate on `serde_json::Value`, delegating to the compiled
`jsonschema::Validator`.
- To validate a binary buffer end-to-end, the consumer reads bytes into
a `Value` tree via the data-access layer, then calls
`validate_json(&Value)`**two explicit steps**. The engine does not
collapse them.
This split is correct as a separation of concerns (ADR-004): the
jsonschema validator operates on `Value`, not raw bytes. But it forces
alkcall to know the two-step dance for its binary frames (channels'
8-byte chunk header) and its JSON payloads (call's `EventEnvelope`)
separately — and to write the bytes-to-`Value` materialization itself,
which today lives only inside `SequentialReader::read_field_value` (an
internal function) and is not exposed for whole-buffer materialization.
alkcall's two roles make this concrete:
1. **channels' chunk header** — `Struct { channel_id: u32 BE, length:
u32 BE }`, packed mode, big-endian. alkcall wants to validate an
incoming 8-byte buffer against this schema in one call. Today it
would have to: construct a `SequentialReader`, walk the two fields,
assemble a `serde_json::Value` from the `FieldValue` enum, then call
`validate_json`.
2. **call's JSON payloads** — `OperationSpec.input_schema` /
`output_schema` / `error_schemas`, plain JSON Schema (no AlkType
kinds). alkcall already validates these via
`jsonschema::Validator` directly; the bytes-to-`Value` step is
`serde_json::from_slice`, which is not alktype's concern.
The "generalized" goal is **one library, two entry points**: a consumer
holding an `AlkTypeEngine` can validate either form without leaving
the engine's API. The two forms share the same underlying
`jsonschema::Validator` (compiled once at load time); they differ only
in whether the input is a `Value` (already parsed) or `&[u8]` (binary
bytes that need to be walked against the layout before validation).
## Decision
**alktype v0.1.0 adds `AlkTypeEngine::validate_bytes(&[u8])` as the
generalized validation entry point for binary buffers.** It collapses
the existing two-step dance (read bytes → `Value`, then validate
`Value`) into one method on the engine.
### API
```rust
impl AlkTypeEngine {
/// Validate a binary buffer against the schema, walking it via the
/// layout engine (packed or aligned) to materialize a `Value` tree,
/// then validating that `Value` against the compiled jsonschema
/// validator. The single-call form of the two-step dance.
///
/// Returns `Ok(())` if the buffer is both layout-valid (fields read
/// without errors) and schema-valid (the materialized `Value`
/// passes the jsonschema validator).
///
/// # Errors
///
/// - `AlkTypeError::Access` if a field cannot be read from the
/// buffer (too short, invalid UTF-8, value out of range for the
/// target type). Carries the field path.
/// - `AlkTypeError::Validation` if the materialized `Value` does
/// not pass the jsonschema validator. Wraps `ValidationError`.
pub fn validate_bytes(&self, buffer: &[u8]) -> Result<(), AlkTypeError>;
// existing — unchanged
pub fn validate_json(&self, instance: &Value) -> Result<(), AlkTypeError>;
pub fn is_valid_json(&self, instance: &Value) -> bool;
}
```
### How it works
`validate_bytes` runs the existing machinery in sequence:
1. **Materialize `Value` from bytes.** A new internal helper
(`materialize_value`, alongside `SequentialReader::read_field_value`
in `src/sequential_reader.rs`) walks the buffer against the schema
and the engine's `Endian`, producing a `serde_json::Value` tree.
Composites are recursed into (`Struct` → object of field values;
`Array` → array of element values; `Union` → dispatch then recurse;
`Record` → object of key/value entries). This is the read phase,
reused from the existing data-access layer; it returns
`AlkTypeError::Access` (with field paths) on read failures.
2. **Validate the `Value`.** The materialized `Value` is passed to the
existing `self.validator.validate(&value)`, producing
`AlkTypeError::Validation` on failure. This is the existing
validate phase, unchanged.
The two phases are the same two steps the consumer would do manually
today. `validate_bytes` is a convenience, not new validation logic. The
underlying `jsonschema::Validator` is unchanged; the data-access read
functions are unchanged.
### Mode dispatch
`validate_bytes` dispatches on `self.mode()`:
- **Packed mode** — walks the buffer with a fresh `SequentialReader`
(the engine is already a reader factory per ADR-007), materializing
fields in declaration order.
- **Aligned mode** — uses the `OffsetMap` to read fields at their
computed offsets, then materializes composites by recursing into the
offset map's nested entries.
Both modes produce the same `Value` form; the validator is
mode-agnostic (it operates on `Value`, not bytes — ADR-004).
### `validate_json` is unchanged
`validate_json(&Value)` and `is_valid_json(&Value)` are unchanged. They
remain the correct entry points for already-parsed JSON (call's
`EventEnvelope` payloads, OperationSpec validation,
TypeBox-produced instances, anything coming off `serde_json::from_slice`
or `serde_json::from_str`). alkcall uses `validate_json` for call's
JSON payloads and `validate_bytes` for channels' binary header — same
engine, two methods, one library.
### What this is not
- **Not a new validation engine.** ADR-001 rejected hand-rolled
validation; `validate_bytes` runs the existing `jsonschema`
validator against the existing materialized `Value`. No new
validator code, no parallel validation path. The "binary-aware
validators that skip the `Value` tree" option we considered is
explicitly rejected — it would re-introduce the hand-rolled engine
that ADR-001 eliminated.
- **Not a `Validator` trait abstraction.** A `Validator` trait with
impls for JSON-only and AlkType-binary schemas was considered and
rejected for v0.1.0. The two impls share little internally
(`validate_json` is a single `jsonschema` call; `validate_bytes` is
materialize + validate), so a trait would add a layer without
unifying behavior. alkcall holds an `AlkTypeEngine` and calls the
method that matches its payload form. If a future consumer needs
type-erasure across heterogeneous validators, a trait can be added
later (two-way door).
- **Not framing-aware.** `validate_bytes` validates the bytes of *one*
schema instance. It does not strip length prefixes, parse
`[length: u32][payload]` framing, or handle multiple frames in a
buffer. That's the consumer's job (alkcall's framing layer). alktype
validates what one schema describes; it does not parse the wire
envelope around it. This matches the framing-outside-alktype decision
for v0.1.0 (see the discussion that produced this ADR).
- **Not a binary-payload validator for JSON-only schemas.**
`validate_bytes` requires the engine's schema to declare `AlkType:*`
kinds — it materializes `Value` via the layout engine, which needs
binary-layout semantics. A pure JSON Schema (call's
`input_schema`, no AlkType kinds) compiled via
`AlkTypeEngine::compile` would fail at the materialize step (no
`AlkType:Struct` at the root). For pure JSON payloads, the consumer
uses `serde_json::from_slice` then `validate_json`. This is the
two-roles split: `validate_bytes` is for binary-layout schemas,
`validate_json` is for JSON payloads.
## Consequences
### Positive
- **alkcall validates both wire forms from one library.** Channels'
chunk header via `validate_bytes(&frame)`, call's JSON payloads via
`validate_json(&Value)`. Same `AlkTypeEngine` (for the binary schema)
or the same `jsonschema::Validator` (for the JSON schema).
- **One call instead of two for binary validation.** The consumer no
longer writes the bytes-to-`Value` materialization step. Today that
step lives only inside `SequentialReader::read_field_value` and is
not exposed for whole-buffer use; `validate_bytes` exposes it via a
single method.
- **Field-path-carrying errors preserved.** Read failures from the
materialize phase carry the field path (ADR-004), so
`validate_bytes` errors are as debuggable as `read_field` errors.
- **Purely additive.** No existing API changes. `validate_json` and
`is_valid_json` are unchanged. Consumers that do the two-step dance
manually today can switch to `validate_bytes` or keep doing it
themselves — the underlying functions are unchanged.
### Negative
- **Materializes a `Value` tree on every call.** `validate_bytes`
builds a `serde_json::Value` from the buffer before validating. For
very large buffers or very hot paths, this allocation is a cost the
two-step dance already pays (the consumer was materializing `Value`
anyway). High-throughput paths that want to skip validation entirely
continue to skip it (ADR-004 — validation is opt-in per operation).
A future "validate bytes without materializing" path is a two-way
door but explicitly out of scope for v0.1.0 (would re-introduce a
hand-rolled validator, ADR-001).
- **Two error variants from one call.** `validate_bytes` can return
`AlkTypeError::Access` (read phase) or `AlkTypeError::Validation`
(validate phase). Consumers that want to distinguish "the bytes are
malformed" from "the bytes are well-formed but the values violate the
schema" can match on the variant. Consumers that don't care treat
both as "validation failed." This is consistent with ADR-004's
single-error-enum approach.
- **Not usable for pure JSON schemas.** A consumer that compiles a
pure JSON Schema (no AlkType kinds) via `AlkTypeEngine::compile` and
then calls `validate_bytes` gets a materialize-phase error. The
method is documented as binary-layout-only. The fix is to use
`validate_json` for JSON payloads — which is what alkcall does for
call's `input_schema`. This is a documentation/expectation issue,
not a correctness issue.
## Scope Boundaries (What This Is Not)
- **No `Validator` trait.** Two methods on one struct, not a trait
abstraction. See "Not a `Validator` trait abstraction" above.
- **No binary-aware validators that skip `Value`.** The
`Value`-materialization path is the validation path. See "Not a new
validation engine" above.
- **No framing parsing.** Length-prefix stripping, multi-frame
buffers, and envelope parsing stay in the consumer. See "Not
framing-aware" above.
- **No `AlkType:Json` kind.** A new kind for "length-prefixed JSON
value validated against a sub-schema" (which would let alkcall
describe a full `[length][JSON payload]` frame as one schema) is
out of scope for v0.1.0. Framing stays outside alktype; alkcall
strips the length prefix and hands the JSON bytes to `serde_json`,
then validates the resulting `Value` with `validate_json`. Revisit
when a concrete consumer needs a single-schema frame description.
## References
- [validation.md](../validation.md) — the existing validation layer
this ADR extends (the §"What validation validates" framing is
preserved; `validate_bytes` is the collapsed form of the two-step
dance it describes)
- [ADR-001](001-alktype-purpose-scope-jsonschema-engine.md) — why
jsonschema not a custom engine; this ADR does not re-introduce one
- [ADR-004](004-error-handling-validation-strategy.md) —
`AlkTypeError` enum, load-time build / access-time check,
field-path-carrying errors; `validate_bytes` inherits all of this
- [ADR-007](007-packed-mode-read-factory.md) — the engine as
`SequentialReader` factory; `validate_bytes` uses this in packed mode
- [ADR-009](009-builder-api.md) — the builder API (companion v0.1.0
addition); alkcall uses the builder to construct the schemas that
`validate_bytes` validates
- `@alkdev/alknet: docs/architecture/crates/channels/channels-wire.md`
— the 8-byte chunk header (the alkcall consumer that needs
`validate_bytes`)

View File

@@ -1,6 +1,6 @@
---
status: draft
last_updated: 2026-07-22
last_updated: 2026-08-11
---
# Open Questions
@@ -59,7 +59,7 @@ Door type is separate from whether a decision is made. A two-way door is a decis
| OQ | Title | Status | Door | Pri |
|----|-------|--------|------|-----|
| [OQ-003](questions/003-builder-api-for-schema-construction.md) | Builder API for Schema Construction | deferred(scope) | two | med |
| [OQ-003](questions/003-builder-api-for-schema-construction.md) | Builder API for Schema Construction | resolved (ADR-009) | two | med |
## Deferred / Blocked
@@ -69,6 +69,11 @@ blocking condition. They are not failures; they are scope management.
This section exists so "what's currently blocking the architect" is
answerable at a glance, not by filtering the tables above.
> **Note**: OQ-003 (Builder API) was resolved by
> [ADR-009](decisions/009-builder-api.md) in v0.1.0 and is retained
> below for traceability — it's marked **RESOLVED**, not deferred.
> The currently-parked OQs are OQ-001 and OQ-002.
### OQ-001: Arrays of Variable-Length-Element Structs
- **Blocked on**: A concrete consumer that needs arrays of structs with
@@ -92,13 +97,17 @@ answerable at a glance, not by filtering the tables above.
- **Priority**: low
- **Full file**: [OQ-002](questions/002-no-std-alloc-support.md)
### OQ-003: Builder API for Schema Construction
### OQ-003: Builder API for Schema Construction — RESOLVED
- **Blocked on**: A concrete need for programmatic schema construction
in Rust. The current consumers (SFTP, metatensor, binary call frames,
TTY negotiation) all have schemas that can be hand-written or
generated from TypeBox. A builder API would be a fluent Rust API that
produces the same JSON Schema structure — it would sit on top of the
engine, not inside it.
- **Priority**: medium
- **Status**: resolved by [ADR-009](decisions/009-builder-api.md) in
v0.1.0.
- **Unblocking condition**: alkcall — the merged `alknet-call` +
`alknet-channels` extraction from `@alkdev/alknet` — is the concrete
consumer that needs programmatic schema construction in Rust. It
builds both binary-layout schemas (channels' 8-byte chunk header)
and JSON payload schemas (`OperationSpec` input/output/error
schemas) at runtime.
- **Resolution**: A fluent Rust builder producing `serde_json::Value`,
covering AlkType kinds and standard JSON Schema. Implemented in
`src/builder.rs`; spec in [builder.md](builder.md). See ADR-009.
- **Full file**: [OQ-003](questions/003-builder-api-for-schema-construction.md)

View File

@@ -1,6 +1,6 @@
---
status: draft
last_updated: 2026-07-22
last_updated: 2026-08-11
---
# alktype — Overview
@@ -115,11 +115,20 @@ schema JSON determines the order of fields in the binary struct.
| Consumer | Schema describes | Engine provides |
|----------|-----------------|-----------------|
| **alkcall** (v0.1.0 first consumer) | channels' `ChunkHeader { channel_id: u32 BE, length: u32 BE }` + call's `OperationSpec.input_schema` / `output_schema` / `error_schemas` | `validate_bytes` for the 8-byte chunk header; `validate_json` for call's JSON payloads; builder API for both |
| russh-sftp | 29 packet structs + Packet union (byte discriminator) | Read/write SFTP frames from bytes |
| metatensor | Model layout (ConvNet struct, tensor refs) | Offset map for mmap'd tensor access |
| binary call frames | `call.requested` / `call.responded` / etc. structs | Read/write binary call frames |
| TTY negotiation | `NegotiateRequest` / `NegotiateResponse` structs | Read/write TTY control frames |
| channels wire | `ChunkHeader { channel_id, length }` | Already trivial (8 bytes, no schema needed) |
| channels wire | `ChunkHeader { channel_id, length }` | 8-byte chunk header (now in scope for alkcall; was "trivial" pre-v0.1.0) |
alkcall — the merged `alknet-call` (call protocol) + `alknet-channels`
(channel multiplexing) extraction from `@alkdev/alknet` — is the
consumer that bumped the builder API (OQ-003) and generalized
validation (ADR-010) into v0.1.0. It uses alktype for two distinct
schema roles (binary layout + JSON payloads) from one library. See
[builder.md](builder.md) and [validation.md](validation.md)
§"validate_bytes".
The russh-sftp case is the most instructive and the highest-value POC
target. The `Packet` enum's `TryFrom<&mut Bytes>` impl is a hand-written
@@ -142,10 +151,13 @@ These boundaries are decided in [ADR-001](decisions/001-alktype-purpose-scope-js
later.
- **Not a code generator.** typebox-rs's `codegen/` module is a separate
concern. The alktype engine consumes schemas; it does not generate them.
- **Not a schema builder.** The alktype engine does not provide a fluent
API for constructing schemas. Schemas are plain JSON — authored in
TypeBox, generated by ujsx components, or hand-written. A builder API
is deferred (OQ-003).
- **Schema builder is in scope as of v0.1.0.** A fluent Rust API for
constructing schemas at runtime, producing `serde_json::Value`, is
shipped in v0.1.0 ([ADR-009](decisions/009-builder-api.md), resolves
OQ-003). The builder covers AlkType kinds and standard JSON Schema;
see [builder.md](builder.md). Schemas may still be authored in
TypeBox, generated by ujsx components, or hand-written — the builder
is an additional construction path, not a replacement.
- **Not a serialization framework.** The alktype engine is not a
general-purpose serde replacement. It operates on raw byte buffers at
computed offsets — no intermediate `Value` tree, no reflection, no
@@ -166,6 +178,11 @@ These boundaries are decided in [ADR-001](decisions/001-alktype-purpose-scope-js
- **[validation.md](validation.md)** — custom keyword validators for all
19 `AlkType:*` kinds, `AlkTypeError`, load-time vs access-time
validation, `AlkTypeEngine` as the compiled form of a schema.
`validate_json` for JSON values; `validate_bytes` for binary buffers
(ADR-010).
- **[builder.md](builder.md)** — fluent Rust API for constructing
alktype JSON Schemas at runtime, producing `serde_json::Value`.
Covers AlkType kinds and standard JSON Schema (ADR-009).
## Design Decisions
@@ -179,6 +196,8 @@ These boundaries are decided in [ADR-001](decisions/001-alktype-purpose-scope-js
| Non-final inline variable fields | [ADR-006](decisions/006-reject-non-final-inline-length-prefixed-in-aligned-mode.md) | Rejected in aligned mode (would clobber subsequent fields) |
| Packed-mode read factory | [ADR-007](decisions/007-packed-mode-read-factory.md) | `engine.sequential_reader()` returns an owned fresh reader |
| TUnion in aligned mode | [ADR-008](decisions/008-reject-tunion-in-aligned-mode.md) | Rejected for v1 (broken semantics; no current consumer needs it) |
| Builder API | [ADR-009](decisions/009-builder-api.md) | Fluent Rust API producing `serde_json::Value`; covers AlkType kinds + standard JSON Schema; resolves OQ-003 |
| Generalized validation — `validate_bytes` | [ADR-010](decisions/010-generalized-validation-validate-bytes.md) | Single-call binary-buffer validation on `AlkTypeEngine`; materialize `Value` from bytes, then validate |
## Open Questions
@@ -186,7 +205,9 @@ See [open-questions.md](open-questions.md) for full details.
- **OQ-001** (deferred(scope)): Arrays of variable-length-element structs.
- **OQ-002** (deferred(scope)): `no_std` + `alloc` support.
- **OQ-003** (deferred(scope)): Builder API for schema construction.
- **OQ-003** (resolved by [ADR-009](decisions/009-builder-api.md)):
Builder API for schema construction. Shipped in v0.1.0; see
[builder.md](builder.md).
## References

View File

@@ -4,23 +4,21 @@
[../overview.md](../overview.md);
`@alkdev/alknet: docs/research/alknet-typedef/findings.md` (the builder
API was noted as the one detail not covered by the POCs)
- **Status**: deferred(scope)
- **Status**: resolved
- **Door type**: Two-way (additive — a builder API can be added without
changing the existing JSON-consumption path)
- **Priority**: medium
- **Impacts**: Blocks programmatic schema construction in Rust without a
JS toolchain. Any consumer that wants to build alktype schemas at
runtime from Rust code (rather than loading pre-authored JSON) must
construct the JSON manually or depend on TypeBox. Does NOT block any
current consumer — all v1 consumers (SFTP, metatensor, binary call
frames, TTY negotiation) use pre-authored schemas.
- **Blocked on**: A concrete need for programmatic schema construction
in Rust. The current consumers (SFTP, metatensor, binary call frames,
TTY negotiation) all have schemas that can be hand-written or generated
from TypeBox.
- **Resolution**: Not yet decidable. The builder API is important but
not needed for the initial consumers. The engine's JSON-consumption
path is the primary interface for v1. A builder API would be a fluent
Rust API that produces the same JSON Schema structure — it would sit
on top of the engine, not inside it.
- **Cross-references**: ADR-001, [schema-layer.md](../schema-layer.md)
- **Impacts**: (resolved) Was: blocks programmatic schema construction
in Rust without a JS toolchain.
- **Blocked on**: (resolved) Was: a concrete need for programmatic
schema construction in Rust.
- **Resolution**: Resolved by [ADR-009](../decisions/009-builder-api.md)
in v0.1.0. alkcall — the merged `alknet-call` + `alknet-channels`
extraction from `@alkdev/alknet` — is the concrete consumer that
unblocked the deferral: it builds binary-layout schemas (channels'
8-byte chunk header) and JSON payload schemas (`OperationSpec`
input/output/error schemas) at runtime from Rust code. The builder
produces `serde_json::Value`, covers both AlkType kinds and standard
JSON Schema, and is implemented in `src/builder.rs`. See
[../builder.md](../builder.md) for the spec.
- **Cross-references**: ADR-001, ADR-009, [../builder.md](../builder.md)

View File

@@ -1,6 +1,6 @@
---
status: draft
last_updated: 2026-07-22
last_updated: 2026-08-11
---
# alktype — Validation
@@ -79,6 +79,9 @@ impl AlkTypeEngine {
pub fn offset_map(&self) -> Option<&OffsetMap>; // Some in aligned mode
pub fn layout_builder(&self) -> Option<&LayoutBuilder>; // Some in packed mode
pub fn sequential_reader(&self) -> Option<SequentialReader>; // owned fresh reader (ADR-007)
pub fn validate_json(&self, instance: &Value) -> Result<(), AlkTypeError>; // ADR-004
pub fn is_valid_json(&self, instance: &Value) -> bool; // ADR-004
pub fn validate_bytes(&self, buffer: &[u8]) -> Result<(), AlkTypeError>; // ADR-010
}
```
@@ -294,6 +297,75 @@ High-throughput paths can skip validation. Security-sensitive paths
(parsing incoming frames from untrusted peers) can validate every frame.
The choice is the consumer's.
### Access time: `engine.validate_bytes(&[u8])` — binary buffer validation
For binary-layout schemas (schemas declaring `AlkType:*` kinds), the
engine offers a single-call form of the two-step dance: walk the bytes
against the layout to materialize a `Value` tree, then validate that
`Value` against the compiled jsonschema validator. Decided in
[ADR-010](decisions/010-generalized-validation-validate-bytes.md).
```rust
pub fn validate_bytes(&self, buffer: &[u8]) -> Result<(), AlkTypeError>;
```
`validate_bytes` runs the existing machinery in sequence:
1. **Materialize `Value` from bytes.** A new internal helper
(`materialize_value`, alongside `SequentialReader::read_field_value`
in `src/sequential_reader.rs`) walks the buffer against the schema
and the engine's `Endian`, producing a `serde_json::Value` tree.
Composites are recursed into (`Struct` → object of field values;
`Array` → array of element values; `Union` → dispatch then recurse;
`Record` → object of key/value entries). The read phase reuses the
existing data-access functions and returns `AlkTypeError::Access`
(with field paths) on read failures.
2. **Validate the `Value`.** The materialized `Value` is passed to the
existing `self.validator.validate(&value)`, producing
`AlkTypeError::Validation` on failure.
Mode dispatch:
- **Packed mode** — walks with a fresh `SequentialReader` (the engine
is already a reader factory per ADR-007), materializing fields in
declaration order.
- **Aligned mode** — uses the `OffsetMap` to read fields at their
computed offsets, then materializes composites by recursing into the
offset map's nested entries.
Both modes produce the same `Value` form; the validator is
mode-agnostic (it operates on `Value`, not bytes — ADR-004).
#### When to use which entry point
| Entry point | Schema form | Input form | When |
|-------------|--------------|------------|------|
| `validate_json(&Value)` | Any (AlkType or plain JSON Schema) | Already-parsed `serde_json::Value` | Call's JSON payloads (`OperationSpec.input_schema`); TypeBox output; anything off `serde_json::from_slice` / `from_str` |
| `validate_bytes(&[u8])` | AlkType binary-layout schema | Raw `&[u8]` buffer | Channels' 8-byte chunk header; future binary call frames; SFTP packet buffers; metatensor index structs |
`validate_bytes` requires the engine's schema to declare `AlkType:*`
kinds — it materializes `Value` via the layout engine, which needs
binary-layout semantics. A pure JSON Schema (call's `input_schema`,
no AlkType kinds) compiled via `AlkTypeEngine::compile` would fail at
the materialize step (no `AlkType:Struct` at the root). For pure JSON
payloads, the consumer uses `serde_json::from_slice` then
`validate_json`. See [ADR-010](decisions/010-generalized-validation-validate-bytes.md)
§"Not a binary-payload validator for JSON-only schemas".
#### What `validate_bytes` is not
- **Not a new validation engine.** It runs the existing `jsonschema`
validator against the existing materialized `Value`. No new
validator code, no parallel validation path (ADR-001).
- **Not framing-aware.** It validates the bytes of *one* schema
instance. It does not strip length prefixes, parse
`[length: u32][payload]` framing, or handle multiple frames in a
buffer. That's the consumer's job. alktype validates what one
schema describes; it does not parse the wire envelope around it.
- **Not a `Validator` trait.** Two methods on one struct, not a trait
abstraction. See [ADR-010](decisions/010-generalized-validation-validate-bytes.md)
§"Not a `Validator` trait abstraction".
## Relationship to Read/Write
Validation and data access are independent operations on the same data.
@@ -315,13 +387,15 @@ representation first, then access the binary buffer.
| Decision | ADR | Summary |
|----------|-----|---------|
| Error handling and validation | [ADR-004](decisions/004-error-handling-validation-strategy.md) | `AlkTypeError` enum; load-time build, access-time check; field-path-carrying errors; jsonschema `ValidationError` wrapping |
| Generalized validation — `validate_bytes` | [ADR-010](decisions/010-generalized-validation-validate-bytes.md) | Single-call binary-buffer validation (materialize `Value` from bytes, then validate); two methods on one struct, not a trait |
| Purpose and scope | [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md) | Why jsonschema not a custom engine |
## Open Questions
None specific to validation. The three alktype OQs (OQ-001, OQ-002,
OQ-003) are about layout, platform support, and schema construction —
not validation.
not validation. OQ-003 is resolved by [ADR-009](decisions/009-builder-api.md);
see [builder.md](builder.md).
## References