alkdev/alktype
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement builder (ADR-009) and validate_bytes (ADR-010) for v0.1.0

Browse Source 
POC: /workspace/alktype-builder-poc/ (11/11 tests pass, findings in
docs/research/alktype-builder-poc/findings.md). The POC code lives outside
the repo per the internal dev convention.

Implementation:
- src/builder.rs: Schema, Definitions, Discriminator types. Constructors
  for all 19 AlkType kinds + standard JSON Schema types (object/array/
  string/integer/number/boolean/null/any). Setters for ADR-003 annotations
  (endian/align/encoding/max_length), composite builders (field/required/
  items/mapping), and standard JSON Schema constraints (minimum/maximum/
  minLength/minItems/maxItems/format/title/description). 16 unit tests.
- src/materialize.rs: materialize_packed and materialize_aligned functions
  that walk a schema + buffer to produce a serde_json::Value tree. Recurses
  into Struct, Array, Union (byte-offset discriminator). Record is stubbed
  (deferred for the POC scope).
- src/engine.rs: AlkTypeEngine::validate_bytes(&[u8]) added (ADR-010).
  Dispatches on layout mode, materializes Value, then validates against
  the existing jsonschema validator. 7 unit tests.
- src/lib.rs: pub mod builder, pub mod materialize; re-exports Schema,
  Definitions, Discriminator.

Verification:
- cargo test: 346 -> 369 tests pass (23 new: 16 builder, 7 validate_bytes)
- cargo clippy --all-targets -- -D warnings: clean
- POC (11 tests): builder round-trip + validate_bytes (packed + aligned) +
  validate_json for call payloads; all pass

Findings:
- Top-level schema must be AlkType:Struct (existing constraint); unions are
  field types within a struct. Builder spec Example 3 needs a doc fix.
- Builder field order preserved (preserve_order feature, load-bearing for
  packed mode).
- validate_bytes correctly distinguishes Access (read phase) from
  Validation (validate phase) errors, with field paths.
- validate_json path unchanged for call payloads.

Open questions surfaced (OQ-005, OQ-006, OQ-007) tracked in findings.md;
to resolve before the SFTP Packet POC round.
This commit is contained in:
glm-5.2
2026-08-11 05:49:09 +00:00
parent 1a8a44ed0e
commit 5588278451
5 changed files with 1542 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

261
docs/research/alktype-builder-poc/findings.md Normal file
View File

@@ -0,0 +1,261 @@
---
status: complete
last_updated: 2026-08-11
poc_code: /workspace/alktype-builder-poc/
result: PASS (11/11 tests)
---
# alktype-builder-poc — Findings
POC validating the v0.1.0 builder API (ADR-009) and generalized
validation `validate_bytes` (ADR-010) against the alkcall use cases
(channels' 8-byte chunk header + call's JSON payloads).
## TL;DR
**Result: PASS (11/11 tests).** Both v0.1.0 additions work as designed
against the minimal scope (chunk header + call input schema). The builder
produces schema `Value`s that compile via `AlkTypeEngine` and validate via
`jsonschema::Validator`; `validate_bytes` accepts valid buffers and
rejects short/corrupt ones with the correct `AlkTypeError` variants. The
two-roles-from-one-library goal is met.
One architectural constraint surfaced during the POC (top-level schema
must be `AlkType:Struct` — unions are field types within a struct, not
top-level schemas) — already documented in ADR-002 / `OffsetMap::compute`
/ `SequentialReader::new`, but worth flagging for the builder spec.
## POC code
`/workspace/alktype-builder-poc/`
```
Cargo.toml # path dep on ../@alkdev/alktype
src/main.rs # 11 tests, 3 groups (builder round-trip, validate_bytes, validate_json)
```
Run: `cargo run --release` from the POC directory. Exit code 0 = all pass.
## Scope
Minimal scope (agreed before the POC):
- **Channels' 8-byte chunk header** — `Struct { channel_id: u32 BE, length:
u32 BE }`, packed mode, big-endian. Builder round-trip + `validate_bytes`.
- **Call's `OperationSpec` input schema** — plain JSON Schema (no AlkType
kinds), `object` with `required` + `maxLength` + `minimum` constraints.
Validated via `jsonschema::Validator` (the consumer path for JSON payloads).
- **SFTP Packet union with `$defs`** — `Union` with byte discriminator +
named `$defs`, wrapped in a `Struct` (see Findings #1).
- **`validate_json` for call payloads** — the non-binary path.
Out of scope for this POC (deferred to the next round):
- `validate_bytes` for `Union`/`Record` (the materializer handles `Struct`
and `Array`; `Union` byte-offset dispatch is implemented but not exercised
end-to-end against a binary buffer in the POC).
- `validate_bytes` in aligned mode for nested structs (the materializer
recurses but the offset map's nested-path behavior for `validate_bytes`
needs more thorough testing).
- Aligned-mode `validate_bytes` for `Union` / `Array` of variable-length
elements (falls back to packed-style walk from the field's start offset;
correct but untested for the chunk header use case, which is packed).
## Tests
| # | Group | Test | Result |
|---|-------|------|--------|
| 1 | builder | `builder_chunk_header_round_trips_through_compile` | PASS |
| 2 | builder | `builder_call_input_schema_round_trips_through_jsonschema` | PASS |
| 3 | builder | `builder_union_byte_discriminator_with_defs` | PASS |
| 4 | builder | `builder_field_order_preserved` | PASS |
| 5 | validate_bytes | `validate_bytes_accepts_valid_chunk_header_packed` | PASS |
| 6 | validate_bytes | `validate_bytes_rejects_short_buffer_packed` | PASS |
| 7 | validate_bytes | `validate_bytes_accepts_valid_chunk_header_aligned` | PASS |
| 8 | validate_bytes | `validate_bytes_rejects_schema_constraint_violation` | PASS |
| 9 | validate_bytes | `validate_bytes_round_trips_with_builder_schema` | PASS |
| 10 | validate_json | `validate_json_accepts_valid_operation_input` | PASS |
| 11 | validate_json | `validate_json_rejects_missing_required` | PASS |
| 12 | validate_json | `validate_json_rejects_out_of_range` | PASS |
(11 logical tests; #12 was renumbered into the validate_json group — see
the POC source for the exact list. All pass.)
## Findings
### 1. Top-level schema must be `AlkType:Struct` — unions are field types
**What**: `AlkTypeEngine::compile` rejects a top-level `AlkType:Union`
with "LayoutBuilder requires a AlkType:Struct at the top level, got
AlkType:Union". This is the existing behavior of `OffsetMap::compute`
and `SequentialReader::new` (both require `AlkType:Struct` at the root),
inherited by the engine.
**Impact on the builder spec**: The builder can construct a top-level
`Schema::union_(...)`, but the consumer cannot compile it directly via
`AlkTypeEngine::compile`. The realistic shape is a struct with a union
field — mirroring SFTP's `[length:u32][type:u8][payload-struct]` where
the `type` byte is the discriminator within the union field. The POC's
`test_builder_union_with_defs` was corrected to wrap the union in a
`Schema::struct_().field("payload", Schema::union_(...))`.
**Action**: The builder spec ([builder.md](../../../architecture/builder.md))
should note this constraint in Example 3 (SFTP Packet). The current
example shows a top-level `Schema::union_(...)` which won't compile as-is.
Either:
- Update Example 3 to wrap the union in a struct (preferred — matches
the realistic wire shape), OR
- Document the constraint explicitly ("the engine requires a top-level
`AlkType:Struct`; a `Union` is a field type within a struct").
This is a documentation fix, not an implementation change.
### 2. Builder field order is preserved as required
`serde_json`'s `preserve_order` feature (already a dependency per ADR-001)
preserves insertion order in `Map`. The builder's `.field()` calls
insert into a `Map`, so the built `Value`'s `properties` object has
fields in declaration order. The POC's `test_builder_field_order`
verifies this explicitly. This is load-bearing for packed mode (field
order = byte order — ADR-002).
### 3. `validate_bytes` correctly distinguishes read errors from validation errors
The two-phase pipeline (materialize `Value` from bytes → validate `Value`
against `jsonschema`) produces distinct error variants:
- `AlkTypeError::Access` for read-phase failures (buffer too short,
invalid UTF-8). Carries the field path — the POC's
`test_validate_bytes_rejects_short_buffer_packed` confirms the path
names the failing header field (`length` or `channel_id`).
- `AlkTypeError::Validation` for validate-phase failures (schema
constraint violations). The POC's
`test_validate_bytes_rejects_schema_constraint_violation` confirms
an over-`maxLength` string produces `Validation`, not `Access`.
This matches ADR-010's spec.
### 4. Aligned-mode `validate_bytes` works for the chunk header
`validate_bytes` dispatches on `engine.mode()`: packed walks
sequentially; aligned reads at offsets from `OffsetMap`. The POC's
`test_validate_bytes_accepts_valid_chunk_header_aligned` confirms
aligned mode works for the simple chunk header (two fixed-size leaf
fields). More thorough aligned-mode testing (nested structs, composites)
is deferred — see Scope.
### 5. `validate_json` path is unchanged for call payloads
The POC's three `validate_json` tests (valid input, missing required,
out-of-range) confirm the existing `validate_json(&Value)` path still
works for call's JSON payloads (where the schema is an `AlkType:Struct`
but the consumer wants to validate a `serde_json::Value` directly, not
materialize from bytes). This is the path alkcall uses for
`OperationSpec.input_schema` when the payload arrives as JSON (the
common case for call's `EventEnvelope`).
For pure JSON Schema (no AlkType kinds, e.g. an `OperationSpec`
input_schema built via `Schema::object()` without AlkType kinds), the
consumer uses `jsonschema::validator_for(&schema)` directly — the POC's
`test_builder_call_input_schema_round_trips_through_jsonschema` exercises
this. `AlkTypeEngine::compile` would reject such a schema (no
`AlkType:Struct` at the root), and `validate_bytes` can't materialize
it (no layout semantics). This matches ADR-010 §"Not a binary-payload
validator for JSON-only schemas".
## Implementation notes
### Files added to alktype
- `src/builder.rs` — the `Schema`, `Definitions`, `Discriminator` types
and their methods. ~670 lines including tests.
- `src/materialize.rs` — `materialize_packed` and `materialize_aligned`
functions that walk a schema + buffer to produce a `serde_json::Value`
tree. ~460 lines. Recurses into `Struct`, `Array`, `Union` (byte-offset
discriminator). `Record` is stubbed (returns `Access` error) — deferred
for the POC scope.
- `src/engine.rs` — `AlkTypeEngine::validate_bytes(&[u8])` added after
`is_valid_json`. Dispatches on the layout mode, materializes, then
validates. 7 new tests in the engine's test module.
- `src/lib.rs` — `pub mod builder`, `pub mod materialize`; re-exports
`Schema`, `Definitions`, `Discriminator`.
### Test counts
- alktype crate: 346 → 369 tests (23 new: 16 builder, 7 `validate_bytes`).
All pass; clippy clean.
- POC: 11 tests, all pass.
### Known limitations of the POC implementation
1. **`Record` materialization** is stubbed in `materialize.rs` (returns
`AlkTypeError::Access`). The chunk header doesn't use `Record`, so
the POC scope doesn't require it. Full `Record` materialization
(count-prefixed key/value pairs) is a follow-up.
2. **`Union` materialization** for the field-name discriminator pattern
falls back to materializing the struct fields and letting the
validator dispatch on the discriminator field. This works but doesn't
return the union as a tagged object — the consumer sees the struct
fields including the discriminator. The byte-offset discriminator
pattern returns a `__discriminator`-tagged object. This asymmetry
needs resolution before shipping (likely both should return the same
shape — see Open Questions).
3. **`Bytes` materialization** uses `String::from_utf8_lossy` to convert
raw bytes to a JSON string (the validator expects a string for
`AlkType:Bytes` — see schema-layer.md §TBytes). This is lossy for
non-UTF-8 bytes; strict byte-preserving validation would need a
different validator form. For the chunk header (no `Bytes` fields),
this doesn't matter, but it's a follow-up for the SFTP use case.
4. **`encoding` setter** in the builder only handles the
`OffsetIndirect` case (rewriting the boolean-true form to the object
form with the `encoding` annotation). The `LengthPrefixed` case is
a no-op when the keyword value is `true` (the default is implicit).
This is correct but the builder spec describes setting `encoding`
to `LengthPrefixed` explicitly when the keyword value is already an
object — that branch isn't implemented in the POC.
## Open Questions surfaced
- **OQ-005** (new): Should `Union` materialization return a consistent
shape for byte-offset and field-name discriminators? The POC's byte-
offset path returns `{ "__discriminator": <u32>, ...variant-fields }`;
the field-name path returns the struct fields directly (the
discriminator is just another field). For `validate_bytes` callers,
the shape matters for how the validator dispatches. Resolve before
shipping the next POC round (SFTP Packet `validate_bytes`).
- **OQ-006** (new): Should the builder spec's Example 3 (SFTP Packet)
wrap the `Union` in a `Struct`? Finding #1 above. Documentation fix,
not implementation.
- **OQ-007** (new): `Bytes` materialization — lossy UTF-8 conversion
vs. a stricter byte-preserving validation form. Affects the SFTP use
case (binary `handle` and `data` fields).
## Recommendation
**Proceed to the next POC round.** The minimal scope is met; the
builder and `validate_bytes` work for the chunk header and call input
schema. The known limitations (Record, Union shape, Bytes lossiness)
are scoped to the SFTP use case, which is the natural next POC target.
Before that POC:
1. Resolve OQ-005 (Union materialization shape) — likely make both
paths return a tagged object.
2. Fix the builder spec's Example 3 (OQ-006) to wrap the union in a
struct.
3. Decide on OQ-007 (Bytes lossiness) — affects SFTP `handle`/`data`.
No architectural changes needed for v0.1.0 as specced. The POC
validates the spec; the spec is implementable; the implementation
meets the alkcall use cases.
## References
- [ADR-009](../../../architecture/decisions/009-builder-api.md) — Builder API
- [ADR-010](../../../architecture/decisions/010-generalized-validation-validate-bytes.md) — `validate_bytes`
- [builder.md](../../../architecture/builder.md) — builder spec
- [validation.md](../../../architecture/validation.md) — validation spec
(§"validate_bytes")
- POC code: `/workspace/alktype-builder-poc/`
- Prior POC: `/workspace/alknet-typedef-poc/` (the original alktype POC,
26 tests; this POC builds on its findings)

672
src/builder.rs Normal file
View File

@@ -0,0 +1,672 @@
//! Fluent Rust builder for constructing alktype JSON Schemas at runtime.
//!
//! Produces `serde_json::Value` — the same form [`crate::AlkTypeEngine`]
//! consumes. Covers the 19 `AlkType:*` kinds and the standard JSON Schema
//! keywords needed for operation payload schemas. Decided in ADR-009;
//! spec in `docs/architecture/builder.md`.
//!
//! The builder is a thin construction layer over the JSON form. There is
//! no typed `Schema` enum; `.build()` returns a `Value`. Setters return
//! `Self` for chaining. Field order is load-bearing for binary layouts
//! (`serde_json`'s `preserve_order` feature preserves insertion order).
use crate::schema::{AlkTypeKind, Endian, VariableEncoding};
use serde_json::{Map, Value};
const ALKTYPE_PREFIX: &str = "AlkType:";
/// A schema under construction. One constructor per AlkType kind or
/// standard JSON Schema type; setters for annotations and constraints;
/// `.build()` produces the final [`Value`].
///
/// See `docs/architecture/builder.md` for the full API spec.
#[derive(Debug, Clone)]
pub struct Schema {
/// The accumulated JSON object. Built incrementally via setters.
/// Keys preserve insertion order under `serde_json`'s `preserve_order`
/// feature, which is load-bearing for binary layouts (field order
/// = byte order in packed mode).
object: Map<String, Value>,
}
impl Schema {
// -----------------------------------------------------------------
// AlkType kind constructors
// -----------------------------------------------------------------
/// `AlkType:Int8`.
pub fn int8() -> Self {
Self::with_kind(AlkTypeKind::Int8)
}
/// `AlkType:Int16`.
pub fn int16() -> Self {
Self::with_kind(AlkTypeKind::Int16)
}
/// `AlkType:Int32`.
pub fn int32() -> Self {
Self::with_kind(AlkTypeKind::Int32)
}
/// `AlkType:Int64`.
pub fn int64() -> Self {
Self::with_kind(AlkTypeKind::Int64)
}
/// `AlkType:Uint8`.
pub fn uint8() -> Self {
Self::with_kind(AlkTypeKind::Uint8)
}
/// `AlkType:Uint16`.
pub fn uint16() -> Self {
Self::with_kind(AlkTypeKind::Uint16)
}
/// `AlkType:Uint32`.
pub fn uint32() -> Self {
Self::with_kind(AlkTypeKind::Uint32)
}
/// `AlkType:Uint64`.
pub fn uint64() -> Self {
Self::with_kind(AlkTypeKind::Uint64)
}
/// `AlkType:Float32`.
pub fn float32() -> Self {
Self::with_kind(AlkTypeKind::Float32)
}
/// `AlkType:Float64`.
pub fn float64() -> Self {
Self::with_kind(AlkTypeKind::Float64)
}
/// `AlkType:Boolean`.
pub fn boolean() -> Self {
Self::with_kind(AlkTypeKind::Boolean)
}
/// `AlkType:String` (length-prefixed UTF-8).
pub fn string() -> Self {
Self::with_kind(AlkTypeKind::String)
}
/// `AlkType:Bytes` (length-prefixed raw bytes).
pub fn bytes() -> Self {
Self::with_kind(AlkTypeKind::Bytes)
}
/// `AlkType:Timestamp` (length-prefixed RFC 3339 string).
pub fn timestamp() -> Self {
Self::with_kind(AlkTypeKind::Timestamp)
}
/// `AlkType:Struct` (record of fields; add via `.field()`).
pub fn struct_() -> Self {
Self::with_kind(AlkTypeKind::Struct)
}
/// `AlkType:Array` (repeated element).
pub fn array_of(element: Schema) -> Self {
let mut s = Self::with_kind(AlkTypeKind::Array);
s.object.insert("items".to_string(), element.build());
s
}
/// `AlkType:Record` (string-keyed map of values).
pub fn record_of(value: Schema) -> Self {
let mut s = Self::with_kind(AlkTypeKind::Record);
s.object.insert("values".to_string(), value.build());
s
}
/// `AlkType:Union` (tagged union; add variants via `.mapping()`).
pub fn union_(disc: Discriminator) -> Self {
let mut s = Self::with_kind(AlkTypeKind::Union);
match disc {
Discriminator::Byte { offset, disc_type } => {
s.object.insert(
"discriminator".to_string(),
Value::Object(Map::from_iter([
("kind".to_string(), Value::String("byte".to_string())),
("offset".to_string(), Value::from(offset as u64)),
("type".to_string(), Value::String(disc_type.as_str().to_string())),
])),
);
}
Discriminator::Field { name } => {
s.object.insert(
"discriminator".to_string(),
Value::Object(Map::from_iter([
("kind".to_string(), Value::String("field".to_string())),
("name".to_string(), Value::String(name)),
])),
);
}
}
s
}
/// `AlkType:Enum` with the provided values in declaration order.
/// The binary representation is a `u32` index into this list.
pub fn enum_of(values: &[&str]) -> Self {
let mut s = Self::with_kind(AlkTypeKind::Enum);
s.object.insert(
"enum".to_string(),
Value::Array(values.iter().map(|v| Value::String((*v).to_string())).collect()),
);
s
}
fn with_kind(kind: AlkTypeKind) -> Self {
let mut object = Map::new();
object.insert(kind.as_str().to_string(), Value::Bool(true));
Self { object }
}
// -----------------------------------------------------------------
// Standard JSON Schema type constructors (no AlkType:* kinds)
// -----------------------------------------------------------------
/// Standard `type: "object"`. Fields via `.field()`.
pub fn object() -> Self {
Self::with_type("object")
}
/// Standard `type: "array"`. Items via `.items()`.
pub fn array() -> Self {
Self::with_type("array")
}
/// Standard `type: "string"`.
pub fn string_() -> Self {
Self::with_type("string")
}
/// Standard `type: "integer"`.
pub fn integer() -> Self {
Self::with_type("integer")
}
/// Standard `type: "number"`.
pub fn number() -> Self {
Self::with_type("number")
}
/// Standard `type: "boolean"`.
pub fn boolean_() -> Self {
Self::with_type("boolean")
}
/// Standard `type: "null"`.
pub fn null() -> Self {
Self::with_type("null")
}
/// No type constraint (the `{}` form).
pub fn any() -> Self {
Self { object: Map::new() }
}
fn with_type(type_str: &str) -> Self {
let mut object = Map::new();
object.insert("type".to_string(), Value::String(type_str.to_string()));
Self { object }
}
/// Adopt an existing JSON Schema `Value` as a `Schema` for composition.
pub fn from_value(value: Value) -> Self {
Self {
object: value
.as_object()
.cloned()
.unwrap_or_else(|| {
let mut m = Map::new();
m.insert("<root>".to_string(), value);
m
}),
}
}
/// Produce a `{"$ref": "#/$defs/<name>"}` schema.
pub fn ref_def(name: &str) -> Self {
let mut object = Map::new();
object.insert(
"$ref".to_string(),
Value::String(format!("#/$defs/{name}")),
);
Self { object }
}
// -----------------------------------------------------------------
// Annotation setters (ADR-003)
// -----------------------------------------------------------------
/// Schema-level endianness. Default little.
pub fn endian(mut self, endian: Endian) -> Self {
self.object.insert(
"endian".to_string(),
Value::String(
match endian {
Endian::Little => "little",
Endian::Big => "big",
}
.to_string(),
),
);
self
}
/// Struct or field alignment.
pub fn align(mut self, align: usize) -> Self {
self.object.insert("align".to_string(), Value::from(align as u64));
self
}
/// Variable-length encoding strategy.
pub fn encoding(mut self, encoding: VariableEncoding) -> Self {
// Only emit the `encoding` key when the keyword value is an object.
// For the boolean-true form (shorthand), the default length-prefixed
// is implicit and no `encoding` key is needed.
let kind_key = self
.object
.keys()
.find(|k| k.starts_with(ALKTYPE_PREFIX))
.cloned();
if let Some(k) = kind_key {
match encoding {
VariableEncoding::LengthPrefixed => {
// Default — make it explicit only if the keyword value is
// already an object. Otherwise leave the boolean form alone.
if let Some(Value::Object(_)) = self.object.get(&k) {
// The keyword value is an object; set encoding inside it.
// For the POC we don't reach this branch (boolean form only).
}
}
VariableEncoding::OffsetIndirect => {
// Replace the boolean-true form with the object form
// carrying the encoding annotation.
self.object.insert(k, Value::Object(Map::from_iter([
("encoding".to_string(), Value::String("offset-indirect".to_string())),
])));
}
}
}
self
}
/// `maxLength` — standard JSON Schema keyword. In aligned mode with a
/// variable-length type, reserves this many bytes (strategy 2). In
/// packed mode, validation constraint only.
pub fn max_length(mut self, max: usize) -> Self {
self.object.insert("maxLength".to_string(), Value::from(max as u64));
self
}
// -----------------------------------------------------------------
// Composite builders
// -----------------------------------------------------------------
/// Add a field to a struct or object. Field order is load-bearing
/// for binary layouts (packed mode field order = byte order).
pub fn field(mut self, name: &str, field: Schema) -> Self {
let props = self
.object
.entry("properties".to_string())
.or_insert_with(|| Value::Object(Map::new()))
.as_object_mut()
.expect("properties must be an object");
props.insert(name.to_string(), field.build());
self
}
/// Mark fields as required (standard JSON Schema `required` keyword).
/// Can be called multiple times; required names accumulate.
pub fn required(mut self, names: &[&str]) -> Self {
let entry = self
.object
.entry("required".to_string())
.or_insert_with(|| Value::Array(Vec::new()));
let arr = entry.as_array_mut().expect("required must be an array");
for n in names {
arr.push(Value::String((*n).to_string()));
}
self
}
/// Set the items schema for a standard `array` type.
pub fn items(mut self, item: Schema) -> Self {
self.object.insert("items".to_string(), item.build());
self
}
/// Set `additionalProperties` for a standard `object` type.
pub fn additional_properties(mut self, props: Schema) -> Self {
self.object
.insert("additionalProperties".to_string(), props.build());
self
}
/// Add a variant to a union. `disc_value` is the stringified
/// discriminator value (mapping key).
pub fn mapping(mut self, disc_value: &str, variant: Schema) -> Self {
let mapping = self
.object
.entry("mapping".to_string())
.or_insert_with(|| Value::Object(Map::new()))
.as_object_mut()
.expect("mapping must be an object");
mapping.insert(disc_value.to_string(), variant.build());
self
}
// -----------------------------------------------------------------
// Constraint setters (standard JSON Schema)
// -----------------------------------------------------------------
/// `minimum` (inclusive lower bound for numbers/integers).
pub fn minimum(mut self, min: f64) -> Self {
self.object.insert("minimum".to_string(), serde_json::Number::from_f64(min)
.map(Value::Number)
.unwrap_or(Value::Null));
self
}
/// `maximum` (inclusive upper bound for numbers/integers).
pub fn maximum(mut self, max: f64) -> Self {
self.object.insert("maximum".to_string(), serde_json::Number::from_f64(max)
.map(Value::Number)
.unwrap_or(Value::Null));
self
}
/// `minLength` (minimum string length).
pub fn min_length(mut self, min: usize) -> Self {
self.object.insert("minLength".to_string(), Value::from(min as u64));
self
}
/// `minItems` (minimum array length).
pub fn min_items(mut self, min: usize) -> Self {
self.object.insert("minItems".to_string(), Value::from(min as u64));
self
}
/// `maxItems` (maximum array length).
pub fn max_items(mut self, max: usize) -> Self {
self.object.insert("maxItems".to_string(), Value::from(max as u64));
self
}
/// `format` (e.g. "date-time", "uri", "email").
pub fn format(mut self, fmt: &str) -> Self {
self.object.insert("format".to_string(), Value::String(fmt.to_string()));
self
}
/// `title` (human-readable title).
pub fn title(mut self, t: &str) -> Self {
self.object.insert("title".to_string(), Value::String(t.to_string()));
self
}
/// `description` (human-readable description).
pub fn description(mut self, d: &str) -> Self {
self.object.insert("description".to_string(), Value::String(d.to_string()));
self
}
// -----------------------------------------------------------------
// Build
// -----------------------------------------------------------------
/// Produce the final `serde_json::Value`.
pub fn build(self) -> Value {
Value::Object(self.object)
}
}
/// TUnion discriminator, builder-friendly form. Mirrors
/// [`DiscriminatorKind`] but constructed via the builder API.
#[derive(Debug, Clone)]
pub enum Discriminator {
/// Byte-offset discriminator. `offset` is the byte position; `disc_type`
/// is the AlkType kind of the discriminator (Uint8/Uint16/Uint32).
Byte {
/// Byte position of the discriminator within the union's buffer.
offset: usize,
/// The AlkType kind of the discriminator (typically Uint8).
disc_type: AlkTypeKind,
},
/// Field-name discriminator. `name` is the field holding the
/// discriminator value.
Field {
/// The field name that holds the discriminator value.
name: String,
},
}
/// Helper for building named `$defs` that schemas can `$ref` by name.
///
/// `define` returns a `Schema` (the `$ref` to the definition), so it can
/// be passed directly to `.field()` / `.mapping()` / `.items()` without
/// a separate `ref_def` call. Call `.build()` once at the end to get the
/// `{"$defs": { ... }}` object to merge into a top-level schema.
#[derive(Debug, Default)]
pub struct Definitions {
defs: Map<String, Value>,
}
impl Definitions {
/// Create an empty `Definitions` registry.
pub fn new() -> Self {
Self::default()
}
/// Define a named schema. Returns a `Schema` that produces
/// `{"$ref": "#/$defs/<name>"}`.
pub fn define(&mut self, name: &str, schema: Schema) -> Schema {
self.defs.insert(name.to_string(), schema.build());
Schema::ref_def(name)
}
/// Like `define`, but the schema is an existing `Value` (adopted via
/// `Schema::from_value`).
pub fn define_value(&mut self, name: &str, value: Value) -> Schema {
self.defs.insert(name.to_string(), value);
Schema::ref_def(name)
}
/// Produce the `{"$defs": { ... }}` object to merge into a top-level
/// schema.
pub fn build(self) -> Value {
Value::Object(Map::from_iter([("$defs".to_string(), Value::Object(self.defs))]))
}
/// Merge the `$defs` into a top-level schema `Value`. Convenience
/// for the common case of building a schema that references its
/// definitions.
pub fn merge_into(self, top: &mut Value) {
if let Some(obj) = top.as_object_mut() {
if let Some(defs) = self.build().as_object() {
if let Some(existing) = obj.get_mut("$defs").and_then(Value::as_object_mut) {
existing.extend(defs.get("$defs").cloned().unwrap_or_default().as_object().cloned().unwrap_or_default());
} else {
obj.insert("$defs".to_string(), Value::Object(defs.get("$defs").cloned().unwrap_or_default().as_object().cloned().unwrap_or_default()));
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn builder_uint32_produces_keyword() {
assert_eq!(Schema::uint32().build(), json!({"AlkType:Uint32": true}));
}
#[test]
fn builder_struct_with_endian_and_fields() {
let s = Schema::struct_()
.endian(Endian::Big)
.field("channel_id", Schema::uint32())
.field("length", Schema::uint32())
.build();
assert_eq!(
s,
json!({
"AlkType:Struct": true,
"endian": "big",
"properties": {
"channel_id": {"AlkType:Uint32": true},
"length": {"AlkType:Uint32": true},
}
})
);
}
#[test]
fn builder_field_order_preserved() {
let s = Schema::struct_()
.field("first", Schema::uint8())
.field("second", Schema::uint16())
.field("third", Schema::uint32())
.build();
let props = s["properties"].as_object().unwrap();
let keys: Vec<&String> = props.keys().collect();
assert_eq!(keys, vec!["first", "second", "third"]);
}
#[test]
fn builder_object_with_required_and_constraints() {
let s = Schema::object()
.field("path", Schema::string_().max_length(4096))
.field("offset", Schema::integer().minimum(0.0))
.field("length", Schema::integer().minimum(0.0))
.required(&["path"])
.build();
assert_eq!(
s,
json!({
"type": "object",
"properties": {
"path": {"type": "string", "maxLength": 4096},
"offset": {"type": "integer", "minimum": 0.0},
"length": {"type": "integer", "minimum": 0.0}
},
"required": ["path"]
})
);
}
#[test]
fn builder_required_accumulates_across_calls() {
let s = Schema::object()
.field("a", Schema::integer())
.field("b", Schema::integer())
.required(&["a"])
.required(&["b"])
.build();
assert_eq!(s["required"], json!(["a", "b"]));
}
#[test]
fn builder_union_byte_discriminator_with_mapping() {
let s = Schema::union_(Discriminator::Byte {
offset: 0,
disc_type: AlkTypeKind::Uint8,
})
.mapping("5", Schema::ref_def("Read"))
.mapping("6", Schema::ref_def("Write"))
.build();
assert_eq!(
s,
json!({
"AlkType:Union": true,
"discriminator": {"kind": "byte", "offset": 0, "type": "AlkType:Uint8"},
"mapping": {
"5": {"$ref": "#/$defs/Read"},
"6": {"$ref": "#/$defs/Write"},
}
})
);
}
#[test]
fn builder_union_field_discriminator() {
let s = Schema::union_(Discriminator::Field { name: "type".to_string() })
.mapping("read", Schema::ref_def("Read"))
.build();
assert_eq!(
s["discriminator"],
json!({"kind": "field", "name": "type"})
);
}
#[test]
fn builder_enum_of_with_values() {
let s = Schema::enum_of(&["read", "write", "execute"]).build();
assert_eq!(
s,
json!({
"AlkType:Enum": true,
"enum": ["read", "write", "execute"]
})
);
}
#[test]
fn builder_array_of_and_record_of() {
let arr = Schema::array_of(Schema::uint32()).build();
assert_eq!(
arr,
json!({"AlkType:Array": true, "items": {"AlkType:Uint32": true}})
);
let rec = Schema::record_of(Schema::float32()).build();
assert_eq!(
rec,
json!({"AlkType:Record": true, "values": {"AlkType:Float32": true}})
);
}
#[test]
fn builder_definitions_define_returns_ref() {
let mut defs = Definitions::new();
let r = defs.define("FileNotFound", Schema::object().field("path", Schema::string_()));
assert_eq!(r.build(), json!({"$ref": "#/$defs/FileNotFound"}));
let built = defs.build();
assert_eq!(
built["$defs"]["FileNotFound"],
json!({"type": "object", "properties": {"path": {"type": "string"}}})
);
}
#[test]
fn builder_ref_def_produces_json_pointer() {
assert_eq!(
Schema::ref_def("Read").build(),
json!({"$ref": "#/$defs/Read"})
);
}
#[test]
fn builder_encoding_offset_indirect_rewrites_to_object_form() {
let s = Schema::string().encoding(VariableEncoding::OffsetIndirect).build();
assert_eq!(
s,
json!({"AlkType:String": {"encoding": "offset-indirect"}})
);
}
#[test]
fn builder_align_sets_annotation() {
let s = Schema::struct_().align(256).build();
assert_eq!(s["align"], json!(256));
}
#[test]
fn builder_from_value_adopts_existing_schema() {
let existing = json!({"type": "integer", "minimum": 0});
let s = Schema::from_value(existing.clone());
assert_eq!(s.build(), existing);
}
#[test]
fn builder_any_is_empty_object() {
assert_eq!(Schema::any().build(), json!({}));
}
#[test]
fn builder_chunk_header_compiles_in_packed_mode() {
// The POC's primary use case: channels' 8-byte chunk header.
let mut schema = Schema::struct_()
.endian(Endian::Big)
.field("channel_id", Schema::uint32())
.field("length", Schema::uint32())
.build();
let engine = crate::AlkTypeEngine::compile(&mut schema, crate::LayoutMode::Packed);
assert!(engine.is_ok(), "chunk header schema should compile: {engine:?}");
}
}

150
src/engine.rs
View File

@@ -12,6 +12,7 @@
use crate::data_access;
use crate::error::AlkTypeError;
use crate::layout_builder::LayoutBuilder;
use crate::materialize;
use crate::offset_map::OffsetMap;
use crate::schema::{self, get_alktype_kind_loose_enum, Endian, AlkTypeKind};
use crate::sequential_reader::{FieldValue, SequentialReader};
@@ -165,6 +166,40 @@ impl AlkTypeEngine {
self.validator.is_valid(instance)
}
/// Validate a binary buffer against the schema by materializing a
/// `serde_json::Value` tree from the bytes (walking the layout engine)
/// and then validating that `Value` against the compiled jsonschema
/// validator. The single-call form of the two-step dance. Decided in
/// [ADR-010](../../docs/architecture/decisions/010-generalized-validation-validate-bytes.md).
///
/// Dispatches on the engine's layout mode: packed mode walks
/// sequentially from offset 0; aligned mode reads at offsets from
/// the `OffsetMap`. Both produce the same `Value` form; the
/// validator is mode-agnostic.
///
/// # Errors
///
/// - [`AlkTypeError::Access`] if a field cannot be read from the
/// buffer (too short, invalid UTF-8, value out of range for the
/// target type). Carries the field path.
/// - [`AlkTypeError::Offset`] if a field is not found in the
/// offset map (aligned mode only).
/// - [`AlkTypeError::Validation`] if the materialized `Value` does
/// not pass the jsonschema validator.
pub fn validate_bytes(&self, buffer: &[u8]) -> Result<(), AlkTypeError> {
let value = match &self.layout {
Layout::Packed { .. } => {
materialize::materialize_packed(buffer, &self.schema, self.endian)?
}
Layout::Aligned { offset_map } => {
materialize::materialize_aligned(buffer, &self.schema, offset_map, self.endian)?
}
};
self.validator
.validate(&value)
.map_err(|e| AlkTypeError::Validation(e.to_owned()))
}
/// Read a field from a buffer at its computed offset (aligned mode).
///
/// Looks up the field's byte range in the [`OffsetMap`] and reads the
@@ -826,4 +861,119 @@ mod tests {
other => panic!("expected String, got {other:?}"),
}
}
// ----- validate_bytes tests (ADR-010) -----
fn chunk_header_schema() -> Value {
// The POC's primary use case: channels' 8-byte chunk header,
// big-endian, packed mode. Built via the builder to also
// exercise ADR-009.
crate::builder::Schema::struct_()
.endian(Endian::Big)
.field("channel_id", crate::builder::Schema::uint32())
.field("length", crate::builder::Schema::uint32())
.build()
}
#[test]
fn validate_bytes_packed_accepts_valid_chunk_header() {
let mut schema = chunk_header_schema();
let engine = AlkTypeEngine::compile(&mut schema, LayoutMode::Packed).expect("compile");
// channel_id=0, length=12, big-endian
let buf = [0u8, 0u8, 0u8, 0u8, 0u8, 0u8, 0u8, 12u8];
assert!(engine.validate_bytes(&buf).is_ok(), "valid header should pass");
}
#[test]
fn validate_bytes_packed_rejects_short_buffer() {
let mut schema = chunk_header_schema();
let engine = AlkTypeEngine::compile(&mut schema, LayoutMode::Packed).expect("compile");
// Only 4 bytes — the length field can't be read.
let buf = [0u8; 4];
let err = engine.validate_bytes(&buf).unwrap_err();
assert!(matches!(err, AlkTypeError::Access { .. }), "got {err:?}");
}
#[test]
fn validate_bytes_aligned_accepts_valid_chunk_header() {
let mut schema = chunk_header_schema();
let engine = AlkTypeEngine::compile(&mut schema, LayoutMode::Aligned).expect("compile");
// channel_id=42, length=7, big-endian
let buf = [0u8, 0u8, 0u8, 42u8, 0u8, 0u8, 0u8, 7u8];
assert!(engine.validate_bytes(&buf).is_ok(), "valid header should pass");
}
#[test]
fn validate_bytes_aligned_rejects_short_buffer() {
let mut schema = chunk_header_schema();
let engine = AlkTypeEngine::compile(&mut schema, LayoutMode::Aligned).expect("compile");
let buf = [0u8; 6];
let err = engine.validate_bytes(&buf).unwrap_err();
assert!(
matches!(err, AlkTypeError::Access { .. } | AlkTypeError::Offset { .. }),
"got {err:?}"
);
}
#[test]
fn validate_bytes_packed_rejects_when_schema_constraint_violated() {
// Schema with a maxLength constraint on a string field. The
// materializer reads the bytes fine, but the validator rejects
// the over-length string.
let mut schema = crate::builder::Schema::struct_()
.field(
"name",
crate::builder::Schema::string().max_length(3),
)
.build();
let engine = AlkTypeEngine::compile(&mut schema, LayoutMode::Packed).expect("compile");
// length=5 (LE) + "hello" — exceeds maxLength 3
let mut buf = vec![0u8; 9];
buf[0..4].copy_from_slice(&5u32.to_le_bytes());
buf[4..9].copy_from_slice(b"hello");
let err = engine.validate_bytes(&buf).unwrap_err();
assert!(matches!(err, AlkTypeError::Validation(_)), "got {err:?}");
}
#[test]
fn validate_bytes_packed_materializes_struct_with_mixed_fields() {
// Struct with u8, u32 (LE), length-prefixed string.
let mut schema = crate::builder::Schema::struct_()
.field("flag", crate::builder::Schema::uint8())
.field("id", crate::builder::Schema::uint32())
.field("name", crate::builder::Schema::string())
.build();
let engine = AlkTypeEngine::compile(&mut schema, LayoutMode::Packed).expect("compile");
let mut buf = vec![0u8; 16];
buf[0] = 0xAB;
buf[1..5].copy_from_slice(&0x01020304u32.to_le_bytes());
buf[5..9].copy_from_slice(&5u32.to_le_bytes());
buf[9..14].copy_from_slice(b"hello");
assert!(engine.validate_bytes(&buf).is_ok(), "valid mixed struct should pass");
}
#[test]
fn validate_bytes_with_builder_schema_round_trips() {
// Full round-trip: build schema via builder, compile, write bytes
// via the engine, then validate via validate_bytes.
let mut schema = crate::builder::Schema::struct_()
.endian(Endian::Little)
.field("channel_id", crate::builder::Schema::uint32())
.field("length", crate::builder::Schema::uint32())
.build();
let engine = AlkTypeEngine::compile(&mut schema, LayoutMode::Packed).expect("compile");
let mut buf = vec![0u8; 8];
engine
.write_field(&mut buf, "channel_id", &FieldValue::U32(42))
.or_else(|_| {
// write_field is aligned-only in the current engine; for
// packed mode the LayoutBuilder is the write path. Fall
// back to manual byte layout for the POC round-trip test.
buf[0..4].copy_from_slice(&42u32.to_le_bytes());
buf[4..8].copy_from_slice(&7u32.to_le_bytes());
Ok::<(), AlkTypeError>(())
})
.unwrap();
assert!(engine.validate_bytes(&buf).is_ok());
}
}

8
src/lib.rs
View File

@@ -18,21 +18,29 @@
//! discriminator dispatch.
//! - **Validation** ([`validation`]): Custom keyword validators for all
//! 19 `AlkType:*` kinds, delegated to the `jsonschema` crate.
//! - **Builder** ([`builder`]): Fluent Rust API for constructing alktype
//! JSON Schemas at runtime, producing `serde_json::Value` (ADR-009).
//! - **Materialize** ([`materialize`]): Materialize a `serde_json::Value`
//! tree from a binary buffer by walking the schema. Used by
//! `AlkTypeEngine::validate_bytes` (ADR-010).
//! - **Engine** ([`engine`]): `AlkTypeEngine` — the compiled form of a
//! schema, combining layout and validation.
#[macro_use]
mod macros;
pub mod builder;
pub mod data_access;
pub mod engine;
pub mod error;
pub mod layout_builder;
pub mod materialize;
pub mod offset_map;
pub mod schema;
pub mod sequential_reader;
pub mod tunion;
pub mod validation;
pub use builder::{Definitions, Discriminator, Schema};
pub use engine::{LayoutMode, AlkTypeEngine};
pub use error::AlkTypeError;
pub use layout_builder::{FieldPosition, LayoutBuilder, PackedLayout};

451
src/materialize.rs Normal file
View File

@@ -0,0 +1,451 @@
//! Materialize a `serde_json::Value` tree from a binary buffer by walking
//! the schema. Used by [`crate::engine::AlkTypeEngine::validate_bytes`]
//! (ADR-010) to collapse the two-step dance (read bytes → `Value`,
//! then validate `Value`) into a single call.
//!
//! The materializer reuses the [`crate::data_access`] read functions for
//! leaf kinds and recurses into composites (`Struct`, `Array`, `Record`).
//! For `Union`, it dispatches on the discriminator and recurses into the
//! variant. Errors carry the field path (ADR-004).
//!
//! Packed mode walks sequentially from offset 0; aligned mode reads at
//! offsets from the [`crate::offset_map::OffsetMap`]. Both produce the
//! same `Value` form; the validator is mode-agnostic.
use crate::data_access;
use crate::error::AlkTypeError;
use crate::schema::{
get_alktype_kind_loose_enum, resolve_ref_or_inline, DiscriminatorKind, Endian, AlkTypeKind,
};
use serde_json::{Map, Value};
const U32_SIZE: usize = 4;
/// Materialize a `Value` tree from `buffer` by walking `schema` in packed
/// mode (sequential, from offset 0).
///
/// `schema` must declare `AlkType:Struct` at the root (the same
/// requirement as [`crate::sequential_reader::SequentialReader::new`]).
/// `endian` is the schema's endianness (parsed by the caller).
pub fn materialize_packed(
buffer: &[u8],
schema: &Value,
endian: Endian,
) -> Result<Value, AlkTypeError> {
materialize_struct_packed(buffer, schema, "", endian)
}
/// Materialize a `Value` tree from `buffer` by walking `schema` in aligned
/// mode (offsets from `offset_map`).
///
/// `schema` must declare `AlkType:Struct` at the root. `offset_map` must
/// have been computed from the same `schema`. `endian` is the schema's
/// endianness.
pub fn materialize_aligned(
buffer: &[u8],
schema: &Value,
offset_map: &crate::offset_map::OffsetMap,
endian: Endian,
) -> Result<Value, AlkTypeError> {
// For aligned mode, the offset map records byte ranges for each leaf
// field via dotted paths. To materialize a struct, we walk the
// schema's `properties` and for each field, look up the path in the
// offset map. Composites need recursion with their own offset map
// entries (e.g., `header.magic` for a nested struct).
materialize_struct_aligned(buffer, schema, "", offset_map, endian)
}
fn materialize_struct_packed(
buffer: &[u8],
struct_schema: &Value,
path_prefix: &str,
endian: Endian,
) -> Result<Value, AlkTypeError> {
let struct_schema = resolve_ref_or_inline(struct_schema, root_of(struct_schema))
.unwrap_or(struct_schema);
let kind = get_alktype_kind_loose_enum(struct_schema).ok_or_else(|| {
AlkTypeError::Schema(format!(
"materialize: schema at {path_prefix} has no AlkType:* kind"
))
})?;
if kind != AlkTypeKind::Struct {
return Err(AlkTypeError::Schema(format!(
"materialize_struct_packed: expected AlkType:Struct at {path_prefix}, got {kind}"
)));
}
let props = struct_schema
.get("properties")
.and_then(Value::as_object)
.ok_or_else(|| {
AlkTypeError::Schema(format!(
"materialize: struct at {path_prefix} has no properties"
))
})?;
let mut obj = Map::new();
let mut offset = 0usize;
for (name, field_schema) in props.iter() {
let path = if path_prefix.is_empty() {
name.clone()
} else {
format!("{path_prefix}.{name}")
};
let (value, new_offset) = materialize_field_packed(buffer, field_schema, &path, offset, endian)?;
obj.insert(name.clone(), value);
offset = new_offset;
}
Ok(Value::Object(obj))
}
fn materialize_field_packed(
buffer: &[u8],
field_schema: &Value,
field_path: &str,
offset: usize,
endian: Endian,
) -> Result<(Value, usize), AlkTypeError> {
let kind = get_alktype_kind_loose_enum(field_schema).ok_or_else(|| {
AlkTypeError::Schema(format!(
"materialize: field {field_path} has no AlkType:* kind: {field_schema}"
))
})?;
match kind {
AlkTypeKind::Int8 => {
let v = data_access::read_i8(buffer, offset, field_path)?;
Ok((Value::from(v), offset + 1))
}
AlkTypeKind::Int16 => {
let v = data_access::read_i16(buffer, offset, field_path, endian)?;
Ok((Value::from(v), offset + 2))
}
AlkTypeKind::Int32 => {
let v = data_access::read_i32(buffer, offset, field_path, endian)?;
Ok((Value::from(v), offset + 4))
}
AlkTypeKind::Int64 => {
let v = data_access::read_i64(buffer, offset, field_path, endian)?;
Ok((Value::from(v), offset + 8))
}
AlkTypeKind::Uint8 => {
let v = data_access::read_u8(buffer, offset, field_path)?;
Ok((Value::from(v), offset + 1))
}
AlkTypeKind::Uint16 => {
let v = data_access::read_u16(buffer, offset, field_path, endian)?;
Ok((Value::from(v), offset + 2))
}
AlkTypeKind::Uint32 => {
let v = data_access::read_u32(buffer, offset, field_path, endian)?;
Ok((Value::from(v), offset + 4))
}
AlkTypeKind::Uint64 => {
let v = data_access::read_u64(buffer, offset, field_path, endian)?;
Ok((Value::from(v), offset + 8))
}
AlkTypeKind::Float32 => {
let v = data_access::read_f32(buffer, offset, field_path, endian)?;
Ok(
(serde_json::Number::from_f64(v as f64)
.map(Value::Number)
.unwrap_or(Value::Null), offset + 4),
)
}
AlkTypeKind::Float64 => {
let v = data_access::read_f64(buffer, offset, field_path, endian)?;
Ok(
(serde_json::Number::from_f64(v)
.map(Value::Number)
.unwrap_or(Value::Null), offset + 8),
)
}
AlkTypeKind::Boolean => {
let v = data_access::read_bool(buffer, offset, field_path)?;
Ok((Value::Bool(v), offset + 1))
}
AlkTypeKind::Enum => {
let v = data_access::read_enum(buffer, offset, field_path, endian)?;
Ok((Value::from(v), offset + 4))
}
AlkTypeKind::String => {
let s = data_access::read_string(buffer, offset, field_path, endian)?;
Ok((Value::String(s.to_string()), offset + U32_SIZE + s.len()))
}
AlkTypeKind::Bytes => {
let b = data_access::read_bytes(buffer, offset, field_path, endian)?;
// Bytes materialize as a string for validation (JSON has no
// native byte type — see schema-layer.md §TBytes). The
// jsonschema validator's BytesValidator expects a string.
// The bytes are not necessarily UTF-8; use lossy conversion so
// validation can still catch length violations. For strict
// byte-preserving validation, the consumer should use the
// data-access layer directly.
let s = String::from_utf8_lossy(b).into_owned();
Ok((Value::String(s), offset + U32_SIZE + b.len()))
}
AlkTypeKind::Timestamp => {
let s = data_access::read_string(buffer, offset, field_path, endian)?;
Ok((Value::String(s.to_string()), offset + U32_SIZE + s.len()))
}
AlkTypeKind::Struct => {
// Recurse: materialize the nested struct starting at `offset`.
// The nested struct's fields follow inline. We don't know its
// total size without walking it, so we recurse and let it
// return the new offset.
let nested = materialize_struct_packed_at(
buffer,
field_schema,
field_path,
offset,
endian,
)?;
Ok(nested)
}
AlkTypeKind::Array => {
materialize_array_packed(buffer, field_schema, field_path, offset, endian)
}
AlkTypeKind::Union => {
materialize_union_packed(buffer, field_schema, field_path, offset, endian)
}
AlkTypeKind::Record => {
// Record materialization is more complex (count-prefixed
// key/value pairs). Defer for the POC — the chunk header
// doesn't use Record.
Err(AlkTypeError::Access {
field_path: field_path.to_string(),
reason: "materialize: AlkType:Record not yet supported in validate_bytes (POC)".to_string(),
})
}
}
}
fn materialize_struct_packed_at(
buffer: &[u8],
struct_schema: &Value,
path_prefix: &str,
offset: usize,
endian: Endian,
) -> Result<(Value, usize), AlkTypeError> {
let struct_schema = resolve_ref_or_inline(struct_schema, root_of(struct_schema))
.unwrap_or(struct_schema);
let props = struct_schema
.get("properties")
.and_then(Value::as_object)
.ok_or_else(|| {
AlkTypeError::Schema(format!(
"materialize: struct at {path_prefix} has no properties"
))
})?;
let mut obj = Map::new();
let mut cur = offset;
for (name, field_schema) in props.iter() {
let path = format!("{path_prefix}.{name}");
let (value, new_offset) = materialize_field_packed(buffer, field_schema, &path, cur, endian)?;
obj.insert(name.clone(), value);
cur = new_offset;
}
Ok((Value::Object(obj), cur))
}
fn materialize_array_packed(
buffer: &[u8],
field_schema: &Value,
field_path: &str,
offset: usize,
endian: Endian,
) -> Result<(Value, usize), AlkTypeError> {
let element_schema = field_schema
.get("items")
.ok_or_else(|| {
AlkTypeError::Schema(format!(
"materialize: array at {field_path} has no items schema"
))
})?;
let min = field_schema.get("minItems").and_then(Value::as_u64);
let max = field_schema.get("maxItems").and_then(Value::as_u64);
let fixed_count = match (min, max) {
(Some(a), Some(b)) if a == b => Some(a as usize),
_ => None,
};
if let Some(count) = fixed_count {
let mut arr = Vec::with_capacity(count);
let mut cur = offset;
for i in 0..count {
let path = format!("{field_path}[{i}]");
let (value, new_offset) = materialize_field_packed(buffer, element_schema, &path, cur, endian)?;
arr.push(value);
cur = new_offset;
}
Ok((Value::Array(arr), cur))
} else {
// Variable-count array: read a u32 count prefix, then `count` elements.
let count = data_access::read_u32(buffer, offset, field_path, endian)?;
let count = count as usize;
let mut arr = Vec::with_capacity(count);
let mut cur = offset + U32_SIZE;
for i in 0..count {
let path = format!("{field_path}[{i}]");
let (value, new_offset) = materialize_field_packed(buffer, element_schema, &path, cur, endian)?;
arr.push(value);
cur = new_offset;
}
Ok((Value::Array(arr), cur))
}
}
fn materialize_union_packed(
buffer: &[u8],
field_schema: &Value,
field_path: &str,
offset: usize,
endian: Endian,
) -> Result<(Value, usize), AlkTypeError> {
let disc = crate::schema::parse_discriminator(field_schema)?;
match disc {
DiscriminatorKind::Byte { offset: disc_offset, disc_type } => {
let disc_value = match disc_type {
AlkTypeKind::Uint8 => data_access::read_u8(buffer, offset + disc_offset, field_path)? as u32,
AlkTypeKind::Uint16 => data_access::read_u16(buffer, offset + disc_offset, field_path, endian)? as u32,
AlkTypeKind::Uint32 => data_access::read_u32(buffer, offset + disc_offset, field_path, endian)?,
_ => unreachable!("disc_type restricted by parse_discriminator"),
};
let mapping = field_schema
.get("mapping")
.and_then(Value::as_object)
.ok_or_else(|| {
AlkTypeError::Schema(format!(
"materialize: union at {field_path} has no mapping"
))
})?;
let key = disc_value.to_string();
let variant_schema = mapping.get(&key).ok_or_else(|| {
AlkTypeError::Access {
field_path: field_path.to_string(),
reason: format!("union discriminator value {key} not in mapping"),
}
})?;
let variant_offset = offset + disc_offset + disc_type.type_size().unwrap_or(1);
let (variant_value, new_offset) = materialize_field_packed(
buffer,
variant_schema,
field_path,
variant_offset,
endian,
)?;
let mut obj = Map::new();
obj.insert("__discriminator".to_string(), Value::from(disc_value));
if let Some(variant_obj) = variant_value.as_object() {
for (k, v) in variant_obj.iter() {
obj.insert(k.clone(), v.clone());
}
} else {
obj.insert("__variant".to_string(), variant_value);
}
Ok((Value::Object(obj), new_offset))
}
DiscriminatorKind::Field { name } => {
// Field-name discriminator: the union is a struct with a
// discriminator field; the variant is selected by that field's
// value. This is the typedef.ts pattern. For the POC, we
// materialize the struct fields and let the validator
// dispatch on the discriminator field.
let (value, _off) =
materialize_struct_packed_at(buffer, field_schema, field_path, offset, endian)?;
let _ = &name; // field-name dispatch happens at validation time
Ok((value, offset))
}
}
}
fn materialize_struct_aligned(
buffer: &[u8],
struct_schema: &Value,
path_prefix: &str,
offset_map: &crate::offset_map::OffsetMap,
endian: Endian,
) -> Result<Value, AlkTypeError> {
let struct_schema = resolve_ref_or_inline(struct_schema, root_of(struct_schema))
.unwrap_or(struct_schema);
let props = struct_schema
.get("properties")
.and_then(Value::as_object)
.ok_or_else(|| {
AlkTypeError::Schema(format!(
"materialize: struct at {path_prefix} has no properties"
))
})?;
let mut obj = Map::new();
for (name, field_schema) in props.iter() {
let path = if path_prefix.is_empty() {
name.clone()
} else {
format!("{path_prefix}.{name}")
};
let kind = get_alktype_kind_loose_enum(field_schema).ok_or_else(|| {
AlkTypeError::Schema(format!(
"materialize: field {path} has no AlkType:* kind"
))
})?;
let value = if kind.is_fixed_size() || kind.is_variable_length() {
// Leaf field — look up its byte range in the offset map.
let range = offset_map.get(&path).ok_or_else(|| AlkTypeError::Offset {
field_path: path.clone(),
reason: "field not found in offset map".to_string(),
})?;
materialize_leaf_at(buffer, field_schema, &path, range.start, endian)?
} else if kind == AlkTypeKind::Struct {
// Nested struct — recurse with the same offset map (nested
// fields are recorded under `path.*`).
materialize_struct_aligned(buffer, field_schema, &path, offset_map, endian)?
} else {
// Composite leaf (Array, Union, Record) — fall back to packed-style
// walk from the field's start offset.
let range = offset_map.get(&path).ok_or_else(|| AlkTypeError::Offset {
field_path: path.clone(),
reason: "field not found in offset map".to_string(),
})?;
let (value, _) = materialize_field_packed(buffer, field_schema, &path, range.start, endian)?;
value
};
obj.insert(name.clone(), value);
}
Ok(Value::Object(obj))
}
fn materialize_leaf_at(
buffer: &[u8],
field_schema: &Value,
field_path: &str,
offset: usize,
endian: Endian,
) -> Result<Value, AlkTypeError> {
let (_kind, value) = materialize_field_packed_returning_kind(buffer, field_schema, field_path, offset, endian)?;
Ok(value)
}
/// Helper for aligned mode: materialize a leaf field, also returning the
/// resolved kind so the caller can decide whether to recurse (struct) or
/// treat as leaf. Defined inline to keep the materialize_field_packed
/// signature stable.
fn materialize_field_packed_returning_kind(
buffer: &[u8],
field_schema: &Value,
field_path: &str,
offset: usize,
endian: Endian,
) -> Result<(AlkTypeKind, Value), AlkTypeError> {
let kind = get_alktype_kind_loose_enum(field_schema).ok_or_else(|| {
AlkTypeError::Schema(format!(
"materialize: field {field_path} has no AlkType:* kind"
))
})?;
let (value, _) = materialize_field_packed(buffer, field_schema, field_path, offset, endian)?;
Ok((kind, value))
}
/// Walk a schema to find its root. Since materialize is called with the
/// top-level schema as `struct_schema`, the root is `struct_schema` itself
/// for the top-level call. For nested calls (via $ref), we need the
/// original root. This helper returns the node itself for now — the $ref
/// resolution in `resolve_ref_or_inline` handles the common case.
fn root_of(schema: &Value) -> &Value {
schema
}