---
status: accepted
last_updated: 2026-08-15
---

# alktype — Overview

The binary struct engine: a small Rust crate that takes a BAST (Binary
Abstract Syntax Tree) document and produces an offset map, read/write
functions, and validation — all driven by the schema. The schema is the
format definition; the engine is generic.

This document covers the crate's purpose, the "schema is the format"
principle, its dependency edges, consumers, and scope boundaries.
Component details are in the sibling documents.

## What

`alktype` is a library crate that consumes BAST documents and produces
three capabilities:

1. **An offset map** — walks the BAST typed tree, computes byte offsets
   for each field based on type sizes, field order, and alignment.
2. **Read/write functions** — given a `&[u8]` buffer and a field path,
   read the field's bytes at its offset (zero-copy for fixed-size types).
   Given a `&mut [u8]` buffer, write a value at its offset.
3. **Validation** — two validators for two input types:
   - `validate_bytes(&[u8])` uses the BAST-native validator (a recursive
     walker over the BAST type tree) to check the value-domain
      constraints the materializer doesn't (integer ranges, `maxLength`,
      enum index bounds, union variant constraints).
   - `validate_json(&Value)` uses a standard `jsonschema::Validator`
     compiled from a consumer-provided JSON Schema (BAST is not involved
     — BAST describes bytes, not JSON shape).

BAST is a JSON document that describes binary data layouts using a
`kind`-based vocabulary with `$defs`/`$ref` for composition. BAST is
itself a valid JSON Schema instance (it has a meta-schema), making it
self-validating, editor-friendly, and trivially consumable from any
language with a JSON parser. See [ADR-BAST](decisions/bast-bast-format.md)
and [`bast-format.md`](bast-format.md).

The heavy lifting is done by the `jsonschema` crate (the
`validate_json` path and BAST document meta-schema validation) and
`serde_json` (BAST document parsing). The novel code is the offset
computation — a recursive walk of the BAST typed tree that computes
byte positions for each field — and the BAST-native validator — a flat
recursive match over the same tree. See
[ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md)
(purpose/scope) and [ADR-BAST](decisions/bast-bast-format.md) (format).

The crate replaces two prior attempts that built their own jsonschema
engines — typebox-rs (~8,400 lines) and the @alkdev/alktype prototype
(~5,600 lines) — with a BAST parser + an offset map + a BAST-native
validator + the `jsonschema` crate for the JSON-validation path. See
[ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md).

## Why

The crate's purpose is to be a binary struct engine for components that
read or write binary data at computed offsets. Instead of per-protocol
serde structs (russh-sftp's 29 packet types), per-handler wire format
code (TTY's 5-byte format parser), or per-format offset computation
(metatensor's tensor access), all of these become instances of the same
engine with different BAST documents.

The guiding insight:

> **The schema is the format.** A BAST document is both the layout spec
> and the validation spec for bytes. No separate format definition, no
> separate parser, no separate validator. One schema, three uses:
> validate, compute offsets, access data.

This is the convergence of three threads identified in the
call-channels-unification research: the `typedef.ts` schema kinds from
TypeBox, the russh-sftp protocol packets, and the metatensor format. The
common pattern: a schema describes the shape of binary data, and the
binary data is the struct's bytes at computed offsets.

The crate was bumped up in the timeline when the call-channels-unification
research surfaced that channels, TTY, and the binary call protocol are
all variations on the same wire-format family — `[discriminant][length][payload]`.
The alktype engine makes the "channels is call with a binary data plane"
unification concrete: the binary data plane's wire format is the call
protocol's own schema system, just binary-encoded. The `channel_open`
marker says "use binary framing"; the alktype engine says "here's how to
read/write the binary payload."

## The "Schema Is the Format" Principle

A BAST document serves three roles simultaneously:

| Role | Mechanism | When |
|------|-----------|------|
| **Validation spec (bytes)** | BAST-native validator (recursive walker over the BAST type tree) | Load time (parse typed tree), access time (`validate_bytes`) |
| **Validation spec (JSON)** | Standard `jsonschema::Validator` from a consumer-provided JSON Schema | Load time (build validator), access time (`validate_json`) |
| **Layout spec** | Offset computation from type sizes + field order | Load time (build offset map) |
| **Data access** | Read/write at computed offsets | Access time (read field, write field) |

No separate format definition, no separate parser, no separate
validator. The BAST document is the single source of truth for the
binary format. Adding a new field to a protocol is adding an entry to
the BAST `fields` array — the engine computes the new offsets
automatically.

This is the same principle as `#[repr(C)]` struct field access, but at
runtime from a portable JSON document instead of at compile-time from
language-specific annotations. The BAST document is the ABI contract.

## Dependencies

```
alktype
├── jsonschema (v0.46, Draft 2020-12, default-features=false) — validate_json path + BAST meta-schema validation
├── serde_json (with preserve_order)                          — BAST document parsing; mapping iteration order is load-bearing
└── (no tokio, no platform deps)                              — WASM-clean by construction
```

`alktype` is dependency-light: `jsonschema` + `serde_json` only.
No tokio, no platform deps. Compiles to `wasm32-unknown-unknown` for
browser use. The `validate_bytes` path does not touch `jsonschema` for
validation (it uses `jsonschema::ValidationError::custom` only for the
error payload type, D-BAST-009) — a small wasm binary-size win.

`serde_json`'s `preserve_order` feature remains a dependency. Under
BAST, struct field order is explicit (the `fields` array), so layout
correctness no longer depends on it; but `mapping` iteration order and
the `Definitions` `$defs` block order are still load-bearing for the
parser's lazy resolution and the builder's output.

## Consumers

| Consumer | Schema describes | Engine provides |
|----------|-----------------|-----------------|
| **alkcall** (v0.1.0 first consumer) | channels' `ChunkHeader { channel_id: u32 BE, length: u32 BE }` + call's `OperationSpec.input_schema` / `output_schema` / `error_schemas` | `validate_bytes` for the 8-byte chunk header; `validate_json` for call's JSON payloads; builder API for both |
| russh-sftp | 29 packet structs + Packet union (byte discriminator) | Read/write SFTP frames from bytes |
| metatensor | Model layout (ConvNet struct, tensor refs) | Offset map for mmap'd tensor access |
| binary call frames | `call.requested` / `call.responded` / etc. structs | Read/write binary call frames |
| TTY negotiation | `NegotiateRequest` / `NegotiateResponse` structs | Read/write TTY control frames |
| channels wire | `ChunkHeader { channel_id, length }` | 8-byte chunk header (now in scope for alkcall; was "trivial" pre-v0.1.0) |

alkcall — the merged `alknet-call` (call protocol) + `alknet-channels`
(channel multiplexing) extraction from `@alkdev/alknet` — is the
consumer that bumped the builder API (OQ-003) and generalized
validation (ADR-010) into v0.1.0. It uses alktype for two distinct
schema roles (binary layout + JSON payloads) from one library. See
[builder.md](builder.md) and [validation.md](validation.md)
§"validate_bytes".

The russh-sftp case is the most instructive and the highest-value POC
target. The `Packet` enum's `TryFrom<&mut Bytes>` impl is a hand-written
dispatch on a type byte followed by serde deserialization. Under alktype,
the dispatch is a BAST `union` with a byte-offset discriminator — the
schema says "byte 0 is the discriminator, bytes 1..N are the variant
struct." The engine reads the discriminator, looks up the variant
schema, computes offsets, reads fields. Same result, no per-packet-type
code.

## Scope Boundaries (What This Is Not)

These boundaries are decided in
[ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md)
and [ADR-BAST](decisions/bast-bast-format.md).

- **Not metatensor.** alktype is the binary struct *engine*. Metatensor
  is a *format* (8-byte header + JSON header + binary data) that uses the
  alktype engine for its offset computation and tensor access.
- **Not a Value system.** TypeBox's `Value.Diff`, `Value.Migrate`,
  `Value.Convert` — schema evolution — is out of scope for v1. The engine
  should not do anything that explicitly blocks adding a Value system
  later.
- **Not a code generator.** typebox-rs's `codegen/` module is a separate
  concern. The alktype engine consumes BAST documents; it does not
  generate them.
- **Schema builder is in scope as of v0.1.0.** A fluent Rust API for
  constructing BAST documents and standard JSON Schemas at runtime,
  producing `serde_json::Value`, is shipped in v0.1.0
  ([ADR-009](decisions/009-builder-api.md), resolves OQ-003). The
  builder covers BAST kinds (`struct_()`) and standard JSON Schema
  (`object()`); see [builder.md](builder.md). BAST documents may still
  be authored in TypeBox, generated by ujsx components, or hand-written
  — the builder is an additional construction path, not a replacement.
- **Not a serialization framework.** The alktype engine is not a
  general-purpose serde replacement. It operates on raw byte buffers at
  computed offsets — no intermediate `Value` tree (except for the
  `validate_bytes` materialization step), no reflection, no dynamic
  dispatch per field. For JSON data, use serde. For binary data with a
  known schema, use alktype.
- **Not a JSON-payload validator.** BAST describes bytes, not JSON
  shape. `validate_json` validates a JSON `Value` against a
  consumer-provided standard JSON Schema, not against the BAST document.
  See [ADR-VAL-SPLIT](decisions/val-split-two-validator-model.md).

## Architecture (component pointers)

- **[schema-layer.md](schema-layer.md)** — the BAST parser (the typed
  surface every engine module walks), the 18 BAST kinds, the
  `AlkTypeKind` enum, and the foundational annotation types.
- **[`bast-format.md`](bast-format.md)** — the normative BAST format
  specification (meta-schema, TypeRef, examples, validation model).
- **[layout-engine.md](layout-engine.md)** — offset computation, the two
  layout modes (packed sequential vs aligned static), alignment,
  endianness, variable-length field handling.
- **[data-access.md](data-access.md)** — read/write functions, TUnion
  dispatch, field paths, zero-copy access for fixed-size types,
  length-prefix reading for variable-length types.
- **[validation.md](validation.md)** — the two-validator model
  (BAST-native for `validate_bytes`, standard `jsonschema` for
  `validate_json`), `AlkTypeError`, load-time vs access-time validation,
  `AlkTypeEngine` as the compiled form of a BAST document.
- **[builder.md](builder.md)** — fluent Rust API for constructing BAST
  documents and standard JSON Schemas at runtime, producing
  `serde_json::Value`. Covers BAST kinds and standard JSON Schema
  (ADR-009, D-BAST-008).

## Design Decisions

| Decision | ADR | Summary |
|----------|-----|---------|
| Purpose, scope, and the jsonschema engine | [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md) | What the crate is/isn't; why jsonschema not a custom engine; "schema is the format" principle; scope boundaries (format-specific content superseded by ADR-BAST) |
| BAST format | [ADR-BAST](decisions/bast-bast-format.md) | BAST as the schema format; meta-schema, `$defs`/`$ref`/`kind` vocabulary; supersedes ADR-001's format-specific content |
| Two layout modes | [ADR-002](decisions/002-two-layout-modes-packed-vs-aligned.md) | Packed sequential (`LayoutBuilder`/`SequentialReader`) for protocols; aligned static (`OffsetMap`) for mmap formats |
| Schema annotations | [ADR-003](decisions/003-schema-annotations.md) | Endianness (schema-level, default LE), alignment (struct + field-level), encoding (length-prefixed vs offset-indirect), TUnion discriminators (byte-offset vs field-name) — semantics carry forward; location moved to BAST type-level properties |
| Error handling and validation | [ADR-004](decisions/004-error-handling-validation-strategy.md) | `AlkTypeError` enum; load-time build, access-time check; field-path-carrying errors; jsonschema `ValidationError` wrapping (validation-strategy section refined by ADR-VAL-SPLIT) |
| Two-validator model | [ADR-VAL-SPLIT](decisions/val-split-two-validator-model.md) | BAST-native validator for `validate_bytes`; standard `jsonschema::Validator` for `validate_json`; D-BAST-006/007/009 |
| Int64/Uint64 kinds | [ADR-005](decisions/005-int64-uint64-first-class-kinds.md) | 64-bit integers as first-class kinds (SFTP offsets, metatensor data_offsets) |
| Non-final inline variable fields | [ADR-006](decisions/006-reject-non-final-inline-length-prefixed-in-aligned-mode.md) | Rejected in aligned mode (would clobber subsequent fields) |
| Packed-mode read factory | [ADR-007](decisions/007-packed-mode-read-factory.md) | `engine.sequential_reader()` returns an owned fresh reader |
| TUnion in aligned mode | [ADR-008](decisions/008-reject-tunion-in-aligned-mode.md) | Rejected for v1 (broken semantics; no current consumer needs it) |
| Builder API | [ADR-009](decisions/009-builder-api.md) | Fluent Rust API producing `serde_json::Value`; covers BAST kinds + standard JSON Schema; resolves OQ-003 (output format amended to BAST / standard JSON Schema by ADR-BAST) |
| Generalized validation — `validate_bytes` | [ADR-010](decisions/010-generalized-validation-validate-bytes.md) | Single-call binary-buffer validation on `AlkTypeEngine`; materialize `Value` from bytes, then validate (validation step amended to the BAST-native validator by ADR-VAL-SPLIT) |

## Open Questions

See [open-questions.md](open-questions.md) for full details.

- **OQ-001** (deferred(scope)): Arrays of variable-length-element
  structs — BAST arrays require `count` in v1 (D-BAST-004), aligning
  with this deferral.
- **OQ-002** (deferred(scope)): `no_std` + `alloc` support.
- **OQ-003** (resolved by [ADR-009](decisions/009-builder-api.md)):
  Builder API for schema construction. Shipped in v0.1.0; see
  [builder.md](builder.md).

## References

- `@alkdev/alknet: docs/research/alknet-typedef/findings.md` — POC results
  (26 tests passing, two layout modes, TUnion dispatch, endianness)
- `@alkdev/alknet: docs/research/call-channels-unification/findings.md`
  §"alknet-typedef: JSON Schema as the binary struct engine" — the
  origin of this research thread
- `@alkdev/alknet: typebox/example/typedef/typedef.ts` — the TypeBox
  schema kinds (619 lines)
- `@alkdev/alknet: jsonschema/` — the jsonschema crate (v0.46.5, Draft
  2020-12)
- `@alkdev/alknet: alknet-typedef-poc/` — the POC code (disposable)
- `@alkdev/alknet: typebox-rs/` — prior attempt, replaced by alktype
- `@alkdev/alknet: alktype-prototype/` — prior attempt (the @alkdev/alktype prototype; not to be confused with this crate, which reuses the name but is backed by the `jsonschema` crate)
- [BAST pivot research record](../research/bast-pivot.md) — motivation,
  POC scope and result, decisions D-BAST-001..009, risks
- [BAST pivot implementation plan](../plans/bast-implementation.md) —
  ordered implementation steps, semver contract, ADR-sync checklist

> **Note**: The research findings, POC code, and prior-attempt paths above
> refer to the parent `@alkdev/alknet` workspace where this crate originated.
> They are preserved here as historical context for the architectural
> decisions; the artifacts themselves are not part of this standalone repo.