---
status: draft
last_updated: 2026-07-22
---

# alktype — Schema Layer

The schema layer: the 19 `AlkType:*` custom type kinds, their mapping to
Rust types and byte sizes, the `jsonschema` custom keyword integration,
TypeBox interop, and the concrete JSON shapes for schema-level
annotations.

## The 19 AlkType Kinds

These are the custom schema kinds defined in TypeBox's `typedef.ts`
(`@alkdev/alknet: typebox/example/typedef/typedef.ts`, 619 lines) and
ported to Rust via `jsonschema` custom keywords. Each kind carries binary
layout semantics — a known byte size (for fixed-size types) or a known
encoding strategy (for variable-length types).

| Kind | TypeBox key | Rust type | Size | Category |
|------|-------------|-----------|------|----------|
| `TFloat32` | `AlkType:Float32` | `f32` | 4 | fixed |
| `TFloat64` | `AlkType:Float64` | `f64` | 8 | fixed |
| `TInt8` | `AlkType:Int8` | `i8` | 1 | fixed |
| `TInt16` | `AlkType:Int16` | `i16` | 2 | fixed |
| `TInt32` | `AlkType:Int32` | `i32` | 4 | fixed |
| `TInt64` | `AlkType:Int64` | `i64` | 8 | fixed |
| `TUint8` | `AlkType:Uint8` | `u8` | 1 | fixed |
| `TUint16` | `AlkType:Uint16` | `u16` | 2 | fixed |
| `TUint32` | `AlkType:Uint32` | `u32` | 4 | fixed |
| `TUint64` | `AlkType:Uint64` | `u64` | 8 | fixed |
| `TBoolean` | `AlkType:Boolean` | `bool` (0x00=false, 0x01=true) | 1 | fixed |
| `TString` | `AlkType:String` | length-prefixed UTF-8 | variable | variable |
| `TBytes` | `AlkType:Bytes` | length-prefixed raw bytes | variable | variable |
| `TStruct` | `AlkType:Struct` | record of fields | sum of field sizes | composite |
| `TUnion` | `AlkType:Union` | tagged union | discriminator + variant | composite |
| `TArray` | `AlkType:Array` | repeated element | count × element size | composite |
| `TEnum` | `AlkType:Enum` | u32 index into enum values | 4 (fixed) | fixed |
| `TRecord` | `AlkType:Record` | count-prefixed sequence of (key, value) pairs | variable | variable |
| `TTimestamp` | `AlkType:Timestamp` | length-prefixed RFC 3339 string | variable | variable |

`AlkType:Int64` and `AlkType:Uint64` are alktype additions —
TypeBox's `typedef.ts` tops out at 32-bit integers. They are required by
the primary POC targets: SFTP `Read`/`Write` packets have `offset: u64`,
and metatensor `data_offsets` are `u64`. See
[ADR-005](decisions/005-int64-uint64-first-class-kinds.md).

### The `AlkTypeKind` enum

The engine represents the 19 kinds as a Rust enum — `AlkTypeKind` — with
one variant per kind (`AlkTypeKind::Float32`, `AlkTypeKind::Struct`, etc.).
The enum provides compile-time exhaustiveness checking and integer
discriminant dispatch (a jump table) instead of string comparison at
every field access. It is `pub` and re-exported from the crate root.

```rust
pub enum AlkTypeKind {
    Int8, Int16, Int32, Int64,
    Uint8, Uint16, Uint32, Uint64,
    Float32, Float64,
    Boolean, Enum,
    String, Bytes, Timestamp,
    Struct, Union, Array, Record,
}
```

The enum carries the kind's binary-layout metadata as inherent methods:

| Method | Returns | Notes |
|--------|---------|-------|
| `as_str(self)` | `&'static str` | The JSON Schema keyword, e.g. `"AlkType:Uint8"` |
| `type_size(self)` | `Option<usize>` | `Some(N)` for fixed-size kinds; `None` for variable/composite |
| `natural_alignment(self)` | `usize` | 1 for u8/i8/bool, 2 for u16/i16, 4 for u32/i32/f32/enum, 8 for u64/i64/f64, 4 for variable-length (the u32 length prefix), 1 for struct/union/array |
| `is_fixed_size(self)` | `bool` | True for the 12 fixed-size primitive kinds |
| `is_composite(self)` | `bool` | True for Struct, Union, Array, Record |
| `is_variable_length(self)` | `bool` | True for String, Bytes, Timestamp, Record |
| `needs_endian(self)` | `bool` | True for kinds whose read/write takes an `Endian` parameter |

`AlkTypeKind` implements `Display` (renders the keyword string) and
`FromStr` (parses the keyword string back into the variant, returning
`AlkTypeError::Schema` for unknown kinds). The layout engines and the
validator dispatch on the enum, not on strings.

### Fixed-size types

`TFloat32`, `TFloat64`, `TInt8`, `TInt16`, `TInt32`, `TUint8`, `TUint16`,
`TUint32`, `TBoolean`, and `TEnum` have known byte sizes. The offset
computation uses these sizes directly. Read/write is zero-copy pointer
cast for these types.

**`TBoolean` byte representation:** `0x00` = false, `0x01` = true. Other
values are invalid and produce a `AlkTypeError::Access` on read.

**`TEnum` binary representation:** A `u32` index into the enum's declared
values, in declaration order. The first declared value is index 0, the
second is index 1, etc. The enum's values are declared via the standard
JSON Schema `"enum"` keyword (e.g., `"enum": ["read", "write", "execute"]`).
The `AlkType:Enum` custom keyword signals that the type is an enum for
layout purposes; the built-in `enum` keyword provides the value list.

**Design note:** TypeBox's `TEnum` is a string enum (variable-length). The
alktype engine uses a `u32` index instead — a deliberate deviation from
TypeBox fidelity in favor of binary efficiency. Most enums have a small
number of variants (e.g., the call protocol's 5 event types); a `u32`
index is compact, fixed-size, and sufficient for any realistic enum. The
JSON representation (for validation) remains a string; the binary
representation is the `u32` index.
The `u32` index follows the schema's endianness annotation (ADR-003), like
all other fixed-size types. In little-endian mode the index is
`u32::from_le_bytes`; in big-endian mode it is `u32::from_be_bytes`.

### Variable-length types

`TString`, `TBytes`, `TRecord`, and `TTimestamp` have variable byte sizes.
The alktype engine supports three strategies for handling variable-length
types in binary layouts, selected by the `encoding` annotation and the
standard JSON Schema `maxLength` keyword:

| Strategy | Encoding annotation | Layout behavior | Use case |
|----------|-------------------|-----------------|----------|
| **Inline length-prefixed** | `"length-prefixed"` (default) | `[length: u32][data]`; shifts subsequent fields in packed mode | Protocol wire formats (SFTP, channels, TTY) |
| **Fixed-size reservation** | (none — uses `maxLength`) | `[data: maxLength bytes]`, zero-padded; fixed offset in aligned mode | mmap-friendly formats where max size is known (database `VARCHAR(N)` pattern) |
| **Offset indirection** | `"offset-indirect"` | `{offset: u32, length: u32}` pointing into a separate data region | Blob tensors, metatensor variable-length data (the blob tensor pattern) |

**Strategy 1: Inline length-prefixing (default).** The field's fixed
portion is a 4-byte length prefix at a computed offset. The variable data
follows immediately after. In packed sequential mode, the length prefix
determines the position of subsequent fields. In aligned static mode, the
length prefix is at a known offset; the variable data is not included in
the static layout. This is the universal pattern used by channels, SFTP,
TTY, and most binary protocols.

**Strategy 2: Fixed-size reservation.** When a variable-length field
declares `maxLength` (a standard JSON Schema keyword), the engine reserves
`maxLength` bytes at a fixed offset in aligned static mode. Data shorter
than `maxLength` is zero-padded; data longer than `maxLength` is a
validation error. This makes the field fixed-size from the layout
perspective — subsequent fields have known, unchanging offsets. This is
the database `VARCHAR(N)` pattern and the metatensor struct-tensor
pattern for fields with known maximum sizes.

In packed sequential mode, `maxLength` is a validation constraint only —
the engine still uses inline length-prefixing (strategy 1) because
protocols don't benefit from fixed-size reservation.

**Strategy 3: Offset indirection.** The field is a struct
`{offset: u32, length: u32}` at a known position. The consumer provides
the data region separately; the engine reads the offset and length, then
slices the data region. This is the metatensor blob tensor pattern — the
index struct lives in one region, the blob data lives in another. Enables
mmap-friendly random access to variable-length data without parsing
length prefixes and without reserving worst-case space.

**Default strategy selection:**
- In packed sequential mode: always strategy 1 (inline length-prefixing).
  `maxLength` is a validation constraint only.
- In aligned static mode: strategy 2 (fixed-size reservation) if
  `maxLength` is declared; strategy 3 (offset indirection) if
  `"encoding": "offset-indirect"` is declared; strategy 1 (inline
  length-prefixing) otherwise.

**Length prefix endianness:** The 4-byte length prefix (strategies 1 and 3)
respects the schema's `"endian"` annotation (ADR-003). In little-endian
mode, the length is `u32::from_le_bytes`. In big-endian mode, the length
is `u32::from_be_bytes`. This ensures SFTP consumers (big-endian) have
consistent byte order for both field values and length prefixes.

**`TBytes`:** Raw bytes — no UTF-8 constraint. The payload is `&[u8]`.
Otherwise identical to `TString` in layout (same three strategies).

**Design note:** `AlkType:Bytes` is an alktype addition — it does
not exist in TypeBox's `typedef.ts` (which defines 16 kinds). It is
included because raw byte arrays are a common binary protocol primitive
(SFTP data payloads, channels payloads, tensor data) and are semantically
distinct from UTF-8 strings. In the binary representation, TBytes is raw
bytes with no encoding (not base64, not hex). In the JSON representation
(for validation), TBytes is a string (JSON has no native byte type).

**`TRecord`:** A string-keyed map. The value type is declared via the
schema's `"values"` property (e.g., `"values": { "AlkType:Float32": true }`).
Binary layout is a count-prefixed sequence of `(key, value)` pairs:
`[count: u32][key_len: u32][key_bytes][value]...` repeated `count` times.
The count is the number of entries. Each key is a length-prefixed UTF-8
string. Each value is encoded according to its declared `AlkType:*` kind
— a `Record<Uint32>` value is 4 raw bytes; a `Record<String>` value is
itself a length-prefixed string; a `Record<Struct>` value is the struct's
fields laid out inline. There is **no separate `value_len` prefix** —
the value's size is determined by its kind (fixed-size kinds have a
known size; variable-length kinds carry their own length prefix). The
count and key-length prefixes respect the schema's endianness. In
aligned static mode with `maxLength`, the entire record is reserved at
`maxLength` bytes (zero-padded).

**`TTimestamp`:** An RFC 3339 timestamp string (the internet profile of
ISO 8601). Stored as a length-prefixed UTF-8 string (strategy 1) or
fixed-size reservation (strategy 2 with `maxLength`). The data-access
layer treats timestamps as opaque length-prefixed strings — it does not
parse or validate the timestamp format. The jsonschema custom keyword
validator checks RFC 3339 conformance at the JSON level (see
[validation.md](validation.md)).

`TArray` is variable-length when the element type is variable-length or
when the count is not known at schema time. For fixed-size element arrays
with a known count, the size is `element_size × count`.

**`TArray` count declaration:** The array count is declared via the
standard JSON Schema `"minItems"` and `"maxItems"` keywords. When
`minItems == maxItems`, the array has a fixed count known at schema time.
When they differ or are absent, the count is variable and the array uses
a length-prefixed encoding: `[count: u32][element_0]...[element_N]`.
The count prefix respects the schema's endianness.

### Composite types

`TStruct` and `TUnion` are composite — their size is the sum of their
fields' sizes (plus alignment padding in aligned static mode). The offset
computation recurses into their properties.

## Schema-Layer Public API

The `schema` module exposes the foundational types and functions every
other module depends on. These are re-exported from the crate root.

### `get_alktype_kind` vs `get_alktype_kind_loose`

The engine recognizes a `AlkType:*` kind on a schema node two ways,
because the keyword value may be either a boolean (`true`) or an
annotation object (`{ "encoding": "..." }`):

| Function | Recognizes | Returns |
|----------|------------|---------|
| `get_alktype_kind(node) -> Option<&str>` | Boolean form only (`{ "AlkType:String": true }`) | The keyword string, e.g. `"AlkType:String"` |
| `get_alktype_kind_loose(node) -> Option<&str>` | Boolean form **and** object form | The keyword string |
| `get_alktype_kind_enum(node) -> Option<AlkTypeKind>` | Boolean form only | The parsed enum variant |
| `get_alktype_kind_loose_enum(node) -> Option<AlkTypeKind>` | Boolean form **and** object form | The parsed enum variant |

The boolean-form-only functions are used by the validator factories
(which reject the object form as a schema error) and the top-level
kind-check in `OffsetMap::compute` / `LayoutBuilder::new` / `SequentialReader::new`
(which require `AlkType:Struct` at the root). The "loose" variants are
used by the layout engines during field traversal, so that a variable-
length field with an `encoding` annotation (`{ "AlkType:String":
{ "encoding": "offset-indirect" } }`) is still recognized as a `String`.

### Annotation parsers

Each schema-level annotation has a dedicated parser that reads it from a
`serde_json::Value` node and returns a sensible default when absent:

| Function | Annotation | Default |
|----------|------------|---------|
| `parse_endian(node) -> Endian` | `"endian"` | `Endian::Little` |
| `parse_align(node) -> Option<usize>` | `"align"` | `None` |
| `parse_max_length(node) -> Option<usize>` | `"maxLength"` | `None` |
| `parse_encoding(keyword_value) -> VariableEncoding` | `"encoding"` (within the keyword's value object) | `VariableEncoding::LengthPrefixed` |
| `parse_discriminator(node) -> Result<DiscriminatorKind, AlkTypeError>` | `"discriminator"` | (required — returns `AlkTypeError::Schema` if absent) |

### Public enums

```rust
pub enum Endian { Little, Big }
pub enum VariableEncoding { LengthPrefixed, OffsetIndirect }
pub enum DiscriminatorKind {
    Byte { offset: usize, disc_type: AlkTypeKind },
    Field { name: String },
}
```

`DiscriminatorKind::Byte` carries the byte position (`offset`) and the
discriminator's `AlkType:*` kind (`disc_type`, restricted to `Uint8`/
`Uint16`/`Uint32`). `DiscriminatorKind::Field` carries the discriminator
field's name. See [data-access.md](data-access.md) §"TUnion Dispatch" for
how these drive dispatch.

### `$ref` resolution and normalization

| Function | Purpose |
|----------|---------|
| `normalize_refs(schema: &mut Value)` | Walks the schema; rewrites every `"$ref"` whose value is a bare name (no `#` prefix) to `"#/$defs/<name>"`. Idempotent. Runs once at `AlkTypeEngine::compile` time. |
| `resolve_ref(root, ref_path) -> Option<&Value>` | Resolves a JSON Pointer `$ref` (e.g. `"#/$defs/Read"`) against the root schema. |
| `resolve_ref_or_inline(node, root) -> Option<&Value>` | If `node` has a `"$ref"`, resolves it against `root`; otherwise returns `node` itself (it's an inline schema). |

`normalize_refs` bridges TypeBox's bare-name ref output and `jsonschema`'s
JSON Pointer requirement. The layout engines call `resolve_ref_or_inline`
on every `$ref`-bearing node they encounter during traversal.

## jsonschema Custom Keyword Integration

The `jsonschema` crate (v0.46.5, Draft 2020-12) supports custom keywords
via the `with_keyword` API. Each `AlkType:*` kind is registered as a
custom keyword:

```rust
let validator = jsonschema::options()
    .with_keyword("AlkType:Float32", factory)
    .with_keyword("AlkType:Int32", factory)
    .with_keyword("AlkType:Struct", factory)
    // ... all 19 kinds
    .build(&schema)?;
```

The factory closure receives the parent schema object, the keyword's
value, and the schema path — enabling cross-keyword awareness. The
`AlkType:Struct` validator, for example, inspects the parent's
`properties` to validate each field against its declared `AlkType:*` kind.

Each custom keyword implementation is ~10 lines. The `jsonschema` crate
handles all structural validation (object properties, required fields,
array items, enum values) — the custom keywords only need to validate
the leaf type constraints. See [validation.md](validation.md) for the
validator implementations.

This is the same pattern as TypeBox's `TypeRegistry.Set` on the JS side.
Same semantics, different language, same JSON Schema wire format. A
TypeBox schema serialized to JSON feeds into the alktype engine after a
single pre-processing step: normalizing `$ref` values (see below).

## TypeBox Interop

TypeBox modules render to standard JSON Schema under `$defs`. A TypeBox
schema like:

```typescript
const TensorRef = Type.Object({
  dtype: Type.Union([Type.Literal("F32"), Type.Literal("I16")]),
  shape: Type.Array(Type.Number()),
  data_offsets: Type.Tuple([Type.Number(), Type.Number()])
});
```

serialized to JSON is a standard JSON Schema with `type: "object"`,
`properties`, and `required`. That JSON feeds into the alktype engine
after `$ref` normalization. The `AlkType:*` custom keywords are added by
TypeBox's `TypeRegistry.Set` — they appear in the serialized JSON as
additional properties on the schema object.

### `$ref` normalization

TypeBox generates bare-name `$ref` values (e.g., `"$ref": "Read"`),
referencing sibling definitions within the same `$defs` block. The
`jsonschema` crate requires full JSON Pointer paths (e.g.,
`"$ref": "#/$defs/Read"`). The alktype engine normalizes TypeBox-style
refs at schema load time via [`normalize_refs`](#ref-resolution-and-normalization)
— a ~20-line recursive walk that rewrites every bare-name `"$ref"` to
`"#/$defs/<name>"`. The normalization is idempotent — full JSON Pointer
refs pass through unchanged. It runs once at `AlkTypeEngine::compile`
time, before the schema is passed to `jsonschema` or the offset
computation.

**Verification:** The jsonschema crate (v0.46.5) rejects bare-name refs
with `Resource 'Read' is not present in a registry`. Full JSON Pointer
refs (`#/$defs/Read`) resolve correctly. The normalization step bridges
the gap between TypeBox's output and jsonschema's input.

The alktype engine does not depend on TypeBox or any JS toolchain. It
consumes JSON — whether that JSON was authored in TypeBox, generated by
a ujsx component, or hand-written. The schema is the interface.

## Schema Annotations

Schema-level annotations control binary layout behavior. These are
decided in [ADR-003](decisions/003-schema-annotations.md).

### Endianness

Schema-level annotation with a default of little-endian:

```json
{ "AlkType:Struct": true, "endian": "big", "properties": { ... } }
```

- `"endian": "little"` (default) — read/write in little-endian byte order.
- `"endian": "big"` — read/write in big-endian byte order.
- Applies to the entire schema and all nested types.

### Alignment

Both struct-level and field-level, with field-level overriding:

```json
{
  "AlkType:Struct": true,
  "align": 256,
  "properties": {
    "weight": { "AlkType:Float32": true, "align": 16 }
  }
}
```

- Struct-level `"align"` sets the default for all fields.
- Field-level `"align"` overrides the struct default.
- Default alignment: 1 for u8/i8/bool, 2 for u16/i16, 4 for u32/i32/f32/
  enum, 8 for u64/i64/f64, 4 for variable-length (the u32 length prefix),
  1 for struct/union/array.
- Only meaningful in aligned static mode (ADR-002). Ignored in packed
  sequential mode.

### Variable-length encoding

The alktype engine supports three strategies for variable-length types
(see §Variable-length types above for full details). The strategy is
selected by the `encoding` annotation and the standard JSON Schema
`maxLength` keyword:

```json
// Strategy 1: Inline length-prefixing (default, shorthand)
{ "AlkType:String": true }

// Strategy 1: Explicit inline length-prefixing
{ "AlkType:String": { "encoding": "length-prefixed" } }

// Strategy 2: Fixed-size reservation (uses standard maxLength)
{ "AlkType:String": true, "maxLength": 256 }

// Strategy 3: Offset indirection (opt-in)
{ "AlkType:String": { "encoding": "offset-indirect" } }
```

- `"encoding": "length-prefixed"` (default) — 4-byte length prefix at
  computed offset, variable data follows immediately. Used by protocol
  wire formats.
- `maxLength` (standard JSON Schema keyword) — in aligned static mode,
  reserves `maxLength` bytes at a fixed offset (zero-padded). Makes the
  field fixed-size from the layout perspective. In packed sequential
  mode, `maxLength` is a validation constraint only.
- `"encoding": "offset-indirect"` — field is a struct
  `{offset: u32, length: u32}` pointing into a separate data region.
  The consumer provides the data region separately. Used by metatensor
  blob tensors.
- Applies to all variable-length types: `AlkType:String`, `AlkType:Bytes`,
  `AlkType:Array`, `AlkType:Record`, `AlkType:Timestamp`.

### TUnion discriminators

Two discriminator kinds: byte-offset (protocol dispatch) and field-name
(typedef.ts pattern).

**Byte-offset discriminator** (SFTP type bytes, call protocol event types):

```json
{
  "AlkType:Union": true,
  "discriminator": {
    "kind": "byte",
    "offset": 0,
    "type": "AlkType:Uint8"
  },
  "mapping": {
    "5": { "$ref": "#/$defs/Read" },
    "6": { "$ref": "#/$defs/Write" },
    "101": { "$ref": "#/$defs/Status" }
  }
}
```

- `"offset"` — byte position of the discriminator.
- `"type"` — the `AlkType:*` kind of the discriminator (typically
  `AlkType:Uint8`).
- Mapping keys are stringified integers. The variant struct starts at
  `offset + discriminator_size`.

**Field-name discriminator** (typedef.ts pattern):

```json
{
  "AlkType:Union": true,
  "discriminator": {
    "kind": "field",
    "name": "type"
  },
  "mapping": {
    "read": { "$ref": "#/$defs/Read" },
    "write": { "$ref": "#/$defs/Write" }
  }
}
```

- `"name"` — the field name holding the discriminator value.
- Mapping keys are string values matching the discriminator field's value.
- The discriminator field is just another field in the struct.

Mapping values may be either inline schemas or `$ref` pointers. Both work.

## Design Decisions

| Decision | ADR | Summary |
|----------|-----|---------|
| Schema annotations | [ADR-003](decisions/003-schema-annotations.md) | Concrete JSON shapes for endianness, alignment, encoding, and TUnion discriminators |
| Int64/Uint64 kinds | [ADR-005](decisions/005-int64-uint64-first-class-kinds.md) | 64-bit integers as first-class kinds (required by SFTP offsets and metatensor data_offsets) |
| Purpose and scope | [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md) | Why jsonschema not a custom engine; "schema is the format" principle |

## Open Questions

See [open-questions.md](open-questions.md) for full details.

- **OQ-003** (deferred(scope)): Builder API for schema construction.

## References

- `@alkdev/alknet: typebox/example/typedef/typedef.ts` — the TypeBox
  schema kinds (619 lines)
- `@alkdev/alknet: jsonschema/` — the jsonschema crate (v0.46.5, Draft
  2020-12)
- [ADR-003](decisions/003-schema-annotations.md) — schema
  annotation shapes
- [validation.md](validation.md) — custom keyword validator implementations