---
status: accepted
last_updated: 2026-08-15
---

# alktype — Schema Layer

The schema layer: the BAST (Binary Abstract Syntax Tree) format and the
typed parser that the layout engines, materializer, and BAST-native
validator walk. BAST replaces the v0.1.0 `AlkType:*` custom-keyword JSON
Schema format decided in ADR-001; the pivot is recorded in
[ADR-BAST](decisions/bast-bast-format.md) and grounded in
[D-BAST-001..009](../research/bast-pivot.md#decisions).

The **normative format specification** is
[`bast-format.md`](bast-format.md) (meta-schema, TypeRef, examples,
validation model). This document describes the *implementation* — the
typed parser in `src/bast.rs` and the foundational `AlkTypeKind` enum
in `src/schema.rs` — and points at the format spec for shape details.

## The 18 BAST Kinds

BAST uses lowercase `kind` strings (`"uint32"`, `"struct"`, `"union"`,
etc.). The engine represents them as the `AlkTypeKind` Rust enum — one
variant per kind — providing compile-time exhaustiveness checking and
integer-discriminant dispatch (a jump table) instead of string
comparison at every field access.

| BAST kind | `AlkTypeKind` | Rust type | Size | Category |
|-----------|---------------|-----------|------|----------|
| `int8` | `Int8` | `i8` | 1 | fixed |
| `int16` | `Int16` | `i16` | 2 | fixed |
| `int32` | `Int32` | `i32` | 4 | fixed |
| `int64` | `Int64` | `i64` | 8 | fixed |
| `uint8` | `Uint8` | `u8` | 1 | fixed |
| `uint16` | `Uint16` | `u16` | 2 | fixed |
| `uint32` | `Uint32` | `u32` | 4 | fixed |
| `uint64` | `Uint64` | `u64` | 8 | fixed |
| `float32` | `Float32` | `f32` | 4 | fixed |
| `float64` | `Float64` | `f64` | 8 | fixed |
| `bool` | `Boolean` | `bool` (`0x00`=false, `0x01`=true) | 1 | fixed |
| `string` | `String` | length-prefixed UTF-8 | variable | variable |
| `bytes` | `Bytes` | length-prefixed raw bytes | variable | variable |
| `struct` | `Struct` | record of fields | sum of field sizes | composite |
| `union` | `Union` | tagged union | discriminator + variant | composite |
| `array` | `Array` | repeated element | count × element size | composite |
| `enum` | `Enum` | u32 index into enum values | 4 (fixed) | fixed |
| `record` | `Record` | count-prefixed (key, value) pairs | variable | variable |

`int64`/`uint64` are alktype additions — TypeBox's `typedef.ts` tops
out at 32-bit integers. Required by SFTP `Read`/`Write` `offset: u64`
and metatensor `data_offsets`. See
[ADR-005](decisions/005-int64-uint64-first-class-kinds.md).

### The `AlkTypeKind` enum

`src/schema.rs` defines the enum:

```rust
pub enum AlkTypeKind {
    Int8, Int16, Int32, Int64,
    Uint8, Uint16, Uint32, Uint64,
    Float32, Float64,
    Boolean, Enum,
    String, Bytes,
    Struct, Union, Array, Record,
}
```

The enum carries the kind's binary-layout metadata as inherent methods:

| Method | Returns | Notes |
|--------|---------|-------|
| `to_bast_str(self)` | `&'static str` | The lowercase BAST kind string (`"uint32"`) |
| `from_bast_str(s)` | `Result<AlkTypeKind, AlkTypeError>` | Parses a lowercase BAST kind string; `AlkTypeError::Schema` for unknowns |
| `type_size(self)` | `Option<usize>` | `Some(N)` for fixed-size kinds; `None` for variable/composite |
| `natural_alignment(self)` | `usize` | 1 for u8/i8/bool, 2 for u16/i16, 4 for u32/i32/f32/enum, 8 for u64/i64/f64, 4 for variable-length (the u32 length prefix), 1 for struct/union/array |
| `is_fixed_size(self)` | `bool` | True for the 12 fixed-size primitive kinds |
| `is_composite(self)` | `bool` | True for Struct, Union, Array, Record |
| `is_variable_length(self)` | `bool` | True for String, Bytes, Record |
| `needs_endian(self)` | `bool` | True for kinds whose read/write takes an `Endian` parameter |

`AlkTypeKind` implements `Display`, backed by `to_bast_str` so the
layout engines, materializer, validator, and parser surface the
canonical BAST name in error messages. `from_bast_str` is the inverse
and is the dispatch point the BAST parser uses to map a `kind` string
to the enum variant (D-BAST-002).

### Foundational annotation types

`src/schema.rs` also defines the two annotation enums (semantics
unchanged from ADR-003; only their *location* in the document moved —
see [ADR-BAST](decisions/bast-bast-format.md) and
[bast-format.md §Variable-Length Encoding](bast-format.md#variable-length-encoding)):

```rust
pub enum Endian { Little, Big }
pub enum VariableEncoding { LengthPrefixed, OffsetIndirect }
```

The `Discriminator` builder enum lives in
[`src/builder.rs`](../../src/builder.rs) (the builder's domain); the BAST
parser's typed discriminator view is
[`BastDiscriminator`](#the-bast-parser-bast-module).

## The BAST Parser (`bast` module)

`src/bast.rs` is the typed surface over a BAST document. Three
consumers walk the same tree — the layout engines
([`offset_map`](layout-engine.md), [`layout_builder`](layout-engine.md),
[`sequential_reader`](layout-engine.md)), the
[`materialize`](data-access.md) layer, and the
[`bast_validation`](validation.md) validator — so a typed view pays for
itself: each walks matched arms over `BastType` instead of re-parsing
raw JSON at every node. Borrowing (not cloning) the source
`serde_json::Value` keeps the parse allocation-free beyond the small
typed nodes themselves.

### Document shape

Every BAST document has the same top-level shape:

```json
{ "$defs": { "<TypeName>": { ...TypeDef... }, ... } }
```

- The `$defs` block is **required** (D-BAST-003). Single-type documents
  are a special case with one entry.
- The **root type name** is a required parameter to
  `AlkTypeEngine::compile(bast_doc, root_name, mode, ...)` (D-BAST-001).
  It selects which `$defs` entry is the top-level type; convention
  (first entry) is fragile and depends on JSON key order, so an
  explicit parameter is used instead.

See [`bast-format.md`](bast-format.md) for the normative TypeDef shapes
(Struct, Union, Enum, FieldDef, TypeRef) and the meta-schema.

### Typed tree

The parser produces a borrowed typed tree:

| Type | Role |
|------|------|
| `BastDoc<'a>` | The parsed document: the root `Value`, the chosen root name, and the parsed root `BastDef`. Entry point via `BastDoc::new(root, root_name)`. |
| `BastDef<'a>` | A named `$defs` entry — `{ name, kind: BastDefKind, source }`. Only `struct`/`union`/`enum` can live at the top level. |
| `BastDefKind<'a>` | `Struct(BastStruct)` / `Union(BastUnion)` / `Enum(BastEnum)`. |
| `BastStruct<'a>` | `{ endian, align, fields: Vec<BastField>, source }`. Field order is the `fields` array order (BAST design principle #4 — no reliance on `serde_json`'s `preserve_order`). |
| `BastField<'a>` | `{ name, ty: BastType, endian, align, encoding, max_length, source }`. Annotations are field-level properties (ADR-003 semantics, BAST location). |
| `BastUnion<'a>` | `{ endian, discriminator, fields, mapping: Vec<(key, BastType)>, source }`. Variant `$ref`s resolve **lazily** — no compile-time inlining. |
| `BastDiscriminator<'a>` | `Byte { offset, disc_type }` / `Field { name }`. The typed view of the `discriminator` object. |
| `BastEnum<'a>` | `{ values: Vec<&'a str>, source }`. Non-empty (enforced). |
| `BastType<'a>` | A TypeRef — `Primitive(AlkTypeKind)` / `Ref(BastRef)` / `Array(BastArray)` / `Record(BastRecord)` / `Struct(...)` / `Union(...)` / `Enum(...)`. The central mechanism for typing fields, array elements, record values, and union variants. |
| `BastRef<'a>` | A `$ref` restricted to `#/$defs/<name>`. Carries just the name. |
| `BastArray<'a>` | `{ element: Box<BastType>, count, source }`. `count` is required in v1 (D-BAST-004). |
| `BastRecord<'a>` | `{ values: Box<BastType>, source }`. |

All of these are re-exported from the crate root (`pub use bast::{...}`
in `src/lib.rs`).

### `$ref` resolution

BAST `$ref`s are always full JSON Pointers restricted to
`#/$defs/<name>` — no external references, no fragment-only pointers,
no bare names (rejected by the parser). The restriction keeps
resolution a single hash lookup and eliminates the v0.1.0
`normalize_refs` pass that rewrote TypeBox's bare-name refs.

`BastDoc` exposes three resolution helpers:

| Method | Purpose |
|--------|---------|
| `lookup_def(name) -> Result<&'a Value, AlkTypeError>` | The single hash lookup into `$defs`. |
| `resolve_ref(r: &BastRef) -> Result<BastDef, AlkTypeError>` | Resolve a `BastRef` to its `BastDef`. |
| `resolve_typeref(ty: &BastType) -> Result<BastType, AlkTypeError>` | Deref one `$ref` level, or return the inline type unchanged. The composite-walkers call this. |
| `resolve_typeref_as_def(ty, path) -> Result<BastDef, AlkTypeError>` | Resolve a `BastType` to a `BastDef`, wrapping inline composites in a synthetic def. Convenient for the validator/materializer. |

Variant `$ref`s (union `mapping` entries) are resolved **lazily** by
the materializer and validator via these helpers — no
`inline_union_variant_refs` compile step (removed under BAST). The
parser only records the `BastRef` target name.

### Untrusted input

Every path that walks a BAST document returns
`Err(AlkTypeError::Schema)` on a malformed document, never
`panic!`/`unreachable!`/`unwrap` (AGENTS.md §3 — the downstream
`alkcall` consumer accepts schemas from arbitrary internet peers in its
hub/spoke topology). Overflow-safe arithmetic (`checked_add`,
`usize::try_from`) is used for any offset/count cast (AGENTS.md §4).

A malformed document (missing `$defs`, missing `kind`, unknown kind
string, dangling `$ref`, empty `mapping`, non-struct/union/enum at the
top level, a field-name union without a `fields` array, etc.) surfaces
as `AlkTypeError::Schema` with a path-annotated message.

### What the parser does *not* do

- **No meta-schema validation.** `BastDoc::new` parses structurally
  (every reachable def parses to a typed `BastDef`) but does not run
  the BAST meta-schema. Consumers that want full structural validation
  can run the meta-schema via `jsonschema` directly
  ([`BAST_META_SCHEMA`](bast-format.md#the-meta-schema) is re-exported
  from the crate root). The parser's structural checks catch the cases
  that matter for layout/materialize/validate; the meta-schema is the
  authoritative well-formedness check.
- **No eager full-document parse.** Only the root definition and the
  definitions it (transitively) references are parsed eagerly; orphan
  `$defs` entries are not checked. Lazy `$ref` resolution reaches the
  rest at access time.
- **No annotation interpretation.** The parser *records* `endian`/
  `align`/`encoding`/`maxLength` on `BastField`/`BastStruct`; the
  layout engines and validator *interpret* them (ADR-003 semantics).

## Schema Annotations

Annotation *semantics* carry forward unchanged from ADR-003; only their
*location* moved from v0.1.0's custom-keyword objects to BAST
type-level properties. The concrete BAST shapes are in
[`bast-format.md`](bast-format.md):

- [Endianness](bast-format.md#endianness) — struct/union-level `endian`
  with field-level override.
- [Alignment](bast-format.md#alignment) — struct/field-level `align`
  (aligned mode only).
- [Variable-length encoding](bast-format.md#variable-length-encoding) —
  field-level `encoding` and `maxLength`.
- [Union discriminators](bast-format.md#union) — `discriminator` object
  on the union def (`byte` or `field`).

The `maxLength` keyword is *not* a BAST invention — it is the standard
JSON Schema `maxLength`, repurposed as a byte-length cap. In aligned
mode it reserves a fixed-size slot; in packed mode it is a validation
constraint only. It applies to `string`/`bytes` fields only: the parser
rejects it on any other kind (review #006 N3 — elsewhere it was
silently unenforced), and in aligned mode a record reservation was
silently corrupt (review #006 M5). See [bast-format.md §Variable-Length
Encoding](bast-format.md#variable-length-encoding) and
[ADR-003](decisions/003-schema-annotations.md).

## What Was Removed

The v0.1.0 custom-keyword accessor layer was removed in step 8 of the
BAST pivot. The `schema` module retains only the foundational types
(`AlkTypeKind`, `Endian`, `VariableEncoding`, shared constants); the
BAST parser is the typed surface every engine module walks. Removed:

- `get_alktype_kind` / `get_alktype_kind_enum` / `get_alktype_kind_loose`
  / `get_alktype_kind_loose_enum` — superseded by the parser's
  `kind`-string dispatch.
- `normalize_refs` / `inline_union_variant_refs` (+ recursive helpers)
  — BAST refs are always `#/$defs/<name>`; one hash lookup, variant
  refs resolve lazily.
- `parse_encoding` / `parse_align` / `parse_max_length` / `parse_endian`
  — `bast.rs` has its own BAST-property-form copies (internal to the
  parser).
- `parse_discriminator` + `DiscriminatorKind` — replaced by
  `bast::BastDiscriminator`; the builder has its own `Discriminator`
  enum.
- `resolve_ref` / `resolve_ref_or_inline` — replaced by
  `BastDoc::lookup_def` / `resolve_typeref`.
- `FromStr` impl, `as_str`, `Endian::from_schema`, `ALKTYPE_PREFIX`,
  `BYTE_DISCRIMINATOR_TYPES`, and their unit tests.

See [ADR-BAST](decisions/bast-bast-format.md) §"What is removed" and
[`bast-format.md` §What is removed](bast-format.md#what-is-removed).

## Design Decisions

| Decision | ADR | Summary |
|----------|-----|---------|
| BAST format, meta-schema, `$defs`/`$ref`/`kind` vocabulary | [ADR-BAST](decisions/bast-bast-format.md) | Supersedes ADR-001's format-specific content; records D-BAST-001..009 |
| Schema annotations | [ADR-003](decisions/003-schema-annotations.md) | Annotation semantics (carry forward unchanged; only location moves) |
| Int64/Uint64 kinds | [ADR-005](decisions/005-int64-uint64-first-class-kinds.md) | 64-bit integers as first-class kinds (required by SFTP offsets and metatensor data_offsets) |
| Purpose and scope | [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md) | Why jsonschema not a custom engine; "schema is the format" principle (format-specific content superseded by ADR-BAST) |

## Open Questions

See [open-questions.md](open-questions.md) for full details.

- **OQ-001** (deferred(scope)): Arrays of variable-length-element
  structs — BAST arrays require `count` in v1 (D-BAST-004), aligning
  with this deferral.

## References

- [`bast-format.md`](bast-format.md) — the normative BAST format
  specification (meta-schema, TypeRef, examples, validation model)
- [ADR-BAST](decisions/bast-bast-format.md) — the BAST format decision
- [ADR-003](decisions/003-schema-annotations.md) — annotation semantics
- [BAST pivot research record](../research/bast-pivot.md) — motivation,
  POC scope and result, decisions D-BAST-001..009
- [validation.md](validation.md) — the BAST-native validator and the
  `validate_json` JSON-Schema path
- [`src/bast.rs`](../../src/bast.rs) — the parser implementation
- [`src/schema.rs`](../../src/schema.rs) — the `AlkTypeKind` enum and
  foundational annotation types