alktype/docs/architecture/README.md

status, last_updated

status	last_updated
draft	2026-07-22

alktype

The binary struct engine: a small Rust crate that takes a JSON Schema with AlkType:* custom keywords and produces an offset map, read/write functions, and validation — all driven by the schema. The schema is the format definition; the engine is generic.

Documents

Document	Status	Description
overview.md	draft	Crate purpose, "schema is the format" principle, dependencies, consumers, scope boundaries
schema-layer.md	draft	The 19 AlkType:* kinds, jsonschema custom keyword integration, TypeBox interop, schema annotations
layout-engine.md	draft	Offset computation, the two layout modes (packed sequential vs aligned static), alignment, endianness, variable-length handling
data-access.md	draft	Read/write functions, TUnion dispatch, field paths, zero-copy access, length-prefix reading
validation.md	draft	Custom keyword validators for all 19 AlkType:* kinds, AlkTypeError, load-time vs access-time validation, AlkTypeEngine

Applicable ADRs

ADR	Title	Relevance
001	Purpose, Scope, and the jsonschema Engine	What the crate is/isn't; why jsonschema not a custom engine; "schema is the format" principle; scope boundaries
002	Two Layout Modes — Packed Sequential vs Aligned Static	The most important architectural finding; when to use each mode; LayoutBuilder/SequentialReader vs OffsetMap
003	Schema Annotations — Endianness, Alignment, Encoding, TUnion Discriminators	Concrete JSON shapes for all schema-level annotations
004	Error Handling and Validation Strategy	AlkTypeError enum; load-time build, access-time check; field-path-carrying errors
005	Int64/Uint64 as First-Class Kinds	64-bit integers (SFTP offsets, metatensor data_offsets); JSON precision caveat
006	Reject Non-Final Inline Length-Prefixed Variable Fields in Aligned Mode	Prevents silent data corruption (inline variable data clobbering subsequent fields)
007	Packed-Mode Read API — Engine as SequentialReader Factory	engine.sequential_reader() returns an owned reader, not a reference
008	Reject TUnion in Aligned Mode for v1	Unions are the protocol pattern; aligned-mode union semantics were broken

Relevant Open Questions

OQ	Title	Status	Relevance
OQ-001	Arrays of variable-length-element structs	deferred(scope)	Requires lazy walking logic; blocked on a concrete consumer that needs it
OQ-002	no_std + alloc support	deferred(scope)	Target std for v1; blocked on an embedded use case
OQ-003	Builder API for schema construction	deferred(scope)	Schemas are authored in TypeBox or hand-written JSON for v1; blocked on a concrete need

Key Design Principles

1. The schema is the format. A JSON Schema with AlkType:* custom keywords is both the validation spec and the layout spec. No separate format definition, no separate parser, no separate validator. One schema, three uses: validate, compute offsets, access data. See overview.md and ADR-001.

2. jsonschema is the validation engine, not a custom engine. The jsonschema crate (v0.46.5, Draft 2020-12) handles validation with custom keyword support. The novel code is the offset computation, not the validation. This eliminates ~14,000 lines of hand-rolled schema engines (typebox-rs, the @alkimiadev/alktype prototype). See schema-layer.md and ADR-001.

3. Two layout modes for two use cases. Packed sequential (LayoutBuilder/SequentialReader) for protocol wire formats (SFTP, channels, TTY). Aligned static (OffsetMap) for mmap-friendly formats (metatensor). The consumer selects the mode; the schema is the same. See layout-engine.md and ADR-002.

4. Variable-length types default to inline length-prefixing. [length: u32][data] is the universal pattern used by channels, SFTP, TTY, and most binary protocols. Offset indirection (the metatensor blob tensor pattern) is opt-in via the encoding annotation. See layout-engine.md and ADR-003.

5. TUnion supports both byte-offset and field-name discriminators. Byte-offset for protocol dispatch (SFTP type bytes, call protocol event types). Field-name for the typedef.ts string pattern. See data-access.md and ADR-003.

6. Endianness is per-schema, default little-endian. The engine reads the "endian" annotation and byte-swaps accordingly. SFTP consumers specify "endian": "big". See layout-engine.md and ADR-003.

7. Validation is opt-in, built once at load time. The jsonschema validator is compiled once at schema load time. Access-time validation is a fast is_valid() check. High-throughput paths can skip validation; security-sensitive paths can validate every frame. See validation.md and ADR-004.

8. Not a serialization framework. The alktype engine is not a general-purpose serde replacement. It operates on raw byte buffers at computed offsets — no intermediate Value tree, no reflection, no dynamic dispatch per field. For JSON data, use serde. For binary data with a known schema, use alktype. See overview.md and ADR-001.

References

• docs/research/alknet-typedef/findings.md — POC results (26 tests passing, two layout modes, TUnion dispatch, endianness)
• docs/research/call-channels-unification/findings.md §"alknet-typedef: JSON Schema as the binary struct engine" — the origin of this research thread
• /workspace/@alkdev/typebox/example/typedef/typedef.ts — the TypeBox schema kinds (619 lines)
• /workspace/jsonschema/ — the jsonschema crate (v0.46.5, Draft 2020-12)
• /workspace/alknet-typedef-poc/ — the POC code (disposable)
• /workspace/@alkimiadev/typebox-rs/ — prior attempt, replaced by alktype
• /workspace/@alkimiadev/alktype/ — prior attempt (the @alkimiadev/alktype prototype, a handler-registry pattern; not to be confused with this crate, which reuses the name but is backed by the jsonschema crate)