---
status: accepted
last_updated: 2026-08-15
---

# alktype

The binary struct engine: a small Rust crate that takes a BAST (Binary
Abstract Syntax Tree) document and produces an offset map, read/write
functions, and validation — all driven by the schema. The schema is the
format definition; the engine is generic.

## Documents

| Document | Status | Description |
|----------|--------|-------------|
| [overview.md](overview.md) | accepted | Crate purpose, "schema is the format" principle, dependencies, consumers, scope boundaries |
| [`bast-format.md`](bast-format.md) | accepted | **Normative BAST format specification.** Meta-schema, TypeRef, TypeDef shapes (Struct/Union/Enum/FieldDef), examples, validation model. The format the engine consumes. |
| [schema-layer.md](schema-layer.md) | accepted | The BAST parser (`src/bast.rs`) — the typed tree (`BastDoc`/`BastDef`/`BastType`/…) every engine module walks, the 19 BAST kinds, the `AlkTypeKind` enum, and the foundational annotation types. |
| [layout-engine.md](layout-engine.md) | draft | Offset computation, the two layout modes (packed sequential vs aligned static), alignment, endianness, variable-length handling |
| [data-access.md](data-access.md) | draft | Read/write functions, TUnion dispatch, field paths, zero-copy access, length-prefix reading |
| [validation.md](validation.md) | accepted | The two-validator model (BAST-native for `validate_bytes`, standard `jsonschema` for `validate_json`), `AlkTypeError`, load-time vs access-time validation, `AlkTypeEngine` as the compiled form of a BAST document (ADR-010, ADR-VAL-SPLIT). |
| [builder.md](builder.md) | accepted | Fluent Rust API for constructing BAST documents (`struct_()`) and standard JSON Schemas (`object()`) at runtime, producing `serde_json::Value` (ADR-009, D-BAST-008). |

### In-progress work

| Document | Status | Description |
|----------|--------|-------------|
| [BAST pivot — research record](../research/bast-pivot.md) | accepted | Motivation, POC scope and result, decisions D-BAST-001..009, risks for the BAST format pivot. Implemented in steps 1–10. |
| [BAST pivot — implementation plan](../plans/bast-implementation.md) | accepted | Ordered implementation steps, the public-API semver contract, and the ADR-sync checklist for the BAST pivot. Steps 1–10 complete. |

## Applicable ADRs

| ADR | Title | Relevance |
|-----|-------|-----------|
| [001](decisions/001-alktype-purpose-scope-jsonschema-engine.md) | Purpose, Scope, and the jsonschema Engine | What the crate is/isn't; why jsonschema not a custom engine; "schema is the format" principle; scope boundaries. *Format-specific content superseded by ADR-BAST; purpose/scope retained.* |
| [BAST](decisions/bast-bast-format.md) | BAST (Binary Abstract Syntax Tree) as the Schema Format | The BAST format, meta-schema, `$defs`/`$ref`/`kind` vocabulary. Supersedes ADR-001's format-specific content; records D-BAST-001..009. |
| [VAL-SPLIT](decisions/val-split-two-validator-model.md) | Two-Validator Model — BAST-Native for Bytes, Standard jsonschema for JSON | `validate_bytes` uses the BAST-native validator; `validate_json` uses a standard `jsonschema::Validator` from a consumer-provided JSON Schema. Records D-BAST-006/007/009. |
| [002](decisions/002-two-layout-modes-packed-vs-aligned.md) | Two Layout Modes — Packed Sequential vs Aligned Static | The most important architectural finding; when to use each mode; `LayoutBuilder`/`SequentialReader` vs `OffsetMap` (format-agnostic — input format changed, modes didn't) |
| [003](decisions/003-schema-annotations.md) | Schema Annotations — Endianness, Alignment, Encoding, TUnion Discriminators | Annotation *semantics* (carry forward unchanged); annotation *location* moved to BAST type-level properties under the pivot |
| [004](decisions/004-error-handling-validation-strategy.md) | Error Handling and Validation Strategy | `AlkTypeError` enum (shape unchanged, D-BAST-009); load-time build, access-time check; field-path-carrying errors. *Validation-strategy section refined by ADR-VAL-SPLIT.* |
| [005](decisions/005-int64-uint64-first-class-kinds.md) | Int64/Uint64 as First-Class Kinds | 64-bit integers (SFTP offsets, metatensor data_offsets); JSON precision caveat |
| [006](decisions/006-reject-non-final-inline-length-prefixed-in-aligned-mode.md) | Reject Non-Final Inline Length-Prefixed Variable Fields in Aligned Mode | Prevents silent data corruption (inline variable data clobbering subsequent fields) |
| [007](decisions/007-packed-mode-read-factory.md) | Packed-Mode Read API — Engine as SequentialReader Factory | `engine.sequential_reader()` returns an owned reader, not a reference |
| [008](decisions/008-reject-tunion-in-aligned-mode.md) | Reject TUnion in Aligned Mode for v1 | Unions are the protocol pattern; aligned-mode union semantics were broken |
| [009](decisions/009-builder-api.md) | Builder API for Schema Construction | Fluent Rust API producing `serde_json::Value`; covers BAST kinds + standard JSON Schema; resolves OQ-003. *Output format amended to BAST / standard JSON Schema by ADR-BAST.* |
| [010](decisions/010-generalized-validation-validate-bytes.md) | Generalized Validation — `validate_bytes` on `AlkTypeEngine` | Single-call binary-buffer validation; materialize `Value` from bytes, then validate. *Validation step amended to the BAST-native validator by ADR-VAL-SPLIT.* |

## Relevant Open Questions

| OQ | Title | Status | Relevance |
|----|-------|--------|-----------|
| OQ-001 | Arrays of variable-length-element structs | deferred(scope) | Requires lazy walking logic; blocked on a concrete consumer that needs it. BAST arrays require `count` in v1 (D-BAST-004), aligning with this deferral. |
| OQ-002 | `no_std` + `alloc` support | deferred(scope) | Target `std` for v1; blocked on an embedded use case |
| OQ-003 | Builder API for schema construction | resolved (ADR-009) | Shipped in v0.1.0; alkcall is the concrete consumer; see [builder.md](builder.md) |
| OQ-004 | `Discriminator::Field` name — `&str` or `String` | resolved | `String`, for ownership simplicity |
| OQ-005 | `Union` materialization shape — byte-offset vs field-name consistency | resolved | Both kinds return `{ "__discriminator": <value>, ...variant-fields }` |
| OQ-006 | Builder spec Example 3 — wrap `Union` in a `Struct` | resolved | [builder.md](builder.md) Example 3 wraps the union in a `Schema::struct_().field("payload", ...)` |
| OQ-007 | `Bytes` materialization — lossy UTF-8 conversion | resolved | Array of u8: materializer produces `Value::Array` of `Value::Number`; BAST-native validator accepts both `Value::String` and `Value::Array` |
| OQ-008 | `UnionValidator` variant dispatch | resolved | BAST-native validator recurses into the selected variant's BAST definition on `__discriminator` lookup — no custom keywords, no `inline_union_variant_refs` |

## Key Design Principles

1. **The schema is the format.** A BAST document is both the layout
   spec and the validation spec for bytes. No separate format
   definition, no separate parser, no separate validator. One schema,
   three uses: validate, compute offsets, access data. See
   [overview.md](overview.md), [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md),
   and [ADR-BAST](decisions/bast-bast-format.md).

2. **BAST is a JSON Schema dialect, not a custom format.** A BAST
   document is valid JSON conforming to the BAST meta-schema (a
   standard Draft 2020-12 JSON Schema). Any JSON Schema validator can
   check whether a BAST document is well-formed; editors with JSON
   Schema support provide autocomplete for free. See
   [`bast-format.md`](bast-format.md) and
   [ADR-BAST](decisions/bast-bast-format.md).

3. **Two layout modes for two use cases.** Packed sequential
   (`LayoutBuilder`/`SequentialReader`) for protocol wire formats (SFTP,
   channels, TTY). Aligned static (`OffsetMap`) for mmap-friendly formats
   (metatensor). The consumer selects the mode; the BAST document is the
   same. See [layout-engine.md](layout-engine.md) and
   [ADR-002](decisions/002-two-layout-modes-packed-vs-aligned.md).

4. **Variable-length types default to inline length-prefixing.**
   `[length: u32][data]` is the universal pattern used by channels, SFTP,
   TTY, and most binary protocols. Offset indirection (the metatensor
   blob tensor pattern) is opt-in via the `encoding` annotation. See
   [layout-engine.md](layout-engine.md) and
   [ADR-003](decisions/003-schema-annotations.md).

5. **TUnion supports both byte-offset and field-name discriminators.**
   Byte-offset for protocol dispatch (SFTP type bytes, call protocol
   event types). Field-name for the typedef.ts string pattern. See
   [data-access.md](data-access.md) and
   [ADR-003](decisions/003-schema-annotations.md).

6. **Endianness is per-schema, default little-endian.** The engine reads
   the struct-level `"endian"` annotation and byte-swaps accordingly.
   SFTP consumers specify `"endian": "big"`. See
   [layout-engine.md](layout-engine.md) and
   [ADR-003](decisions/003-schema-annotations.md).

7. **Two validators for two input types.** `validate_bytes(&[u8])` uses
   the BAST-native validator (a recursive walker over the BAST type
   tree — no `jsonschema` involvement). `validate_json(&Value)` uses a
   standard `jsonschema::Validator` from a consumer-provided JSON Schema
   (BAST is not involved — BAST describes bytes, not JSON shape). One
   `AlkTypeError::Validation` variant covers both (D-BAST-009). See
   [validation.md](validation.md) and
   [ADR-VAL-SPLIT](decisions/val-split-two-validator-model.md).

8. **Not a serialization framework.** The alktype engine is not a
   general-purpose serde replacement. It operates on raw byte buffers at
   computed offsets — no reflection, no dynamic dispatch per field. For
   JSON data, use serde. For binary data with a known BAST document, use
   alktype. See [overview.md](overview.md) and
   [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md).

9. **Schemas can be built at runtime from Rust.** A fluent builder API
   produces `serde_json::Value` for both BAST documents (`struct_()`) and
   standard JSON Schemas (`object()`), covering alkcall's two roles
   (binary layout + JSON payloads) from one module. The builder is
   additive — consumers with static BAST documents continue to load
   JSON. See [builder.md](builder.md) and
   [ADR-009](decisions/009-builder-api.md).

10. **Two validation entry points, one engine.** `validate_json(&Value)`
    for already-parsed JSON (call's payloads); `validate_bytes(&[u8])`
    for binary buffers (channels' chunk header). Different validators,
    one `AlkTypeError::Validation` variant. See [validation.md](validation.md),
    [ADR-010](decisions/010-generalized-validation-validate-bytes.md),
    and [ADR-VAL-SPLIT](decisions/val-split-two-validator-model.md).

## References

- `@alkdev/alknet: docs/research/alknet-typedef/findings.md` — POC results
  (26 tests passing, two layout modes, TUnion dispatch, endianness)
- `@alkdev/alknet: docs/research/call-channels-unification/findings.md`
  §"alknet-typedef: JSON Schema as the binary struct engine" — the
  origin of this research thread
- `@alkdev/alknet: typebox/example/typedef/typedef.ts` — the TypeBox
  schema kinds (619 lines)
- `@alkdev/alknet: jsonschema/` — the jsonschema crate (v0.46.5, Draft
  2020-12)
- `@alkdev/alknet: alknet-typedef-poc/` — the POC code (disposable)
- `@alkdev/alknet: typebox-rs/` — prior attempt, replaced by alktype
- `@alkdev/alknet: alktype-prototype/` — prior attempt (the @alkdev/alktype prototype, a handler-registry pattern; not to be confused with this crate, which reuses the name but is backed by the `jsonschema` crate)

> **Note**: The research findings, POC code, and prior-attempt paths above
> refer to the parent `@alkdev/alknet` workspace where this crate originated.
> They are preserved here as historical context for the architectural
> decisions; the artifacts themselves are not part of this standalone repo.