Rebrand alknet-typedef to alktype in docs, crate name, and lib name
Renumber ADRs 095-102 to 001-008 and OQs 069-071 to 001-003, and update all cross-references (titles, body prose, file-path links, tables) across the 5 spec docs, README, open-questions index, and all 11 ADR/OQ files. Inline the ADR-009 door-type definition from the parent alknet project (broken cross-project reference). Rebrand prose: alknet-typedef -> alktype in headings, body text, dependency diagrams, and "additions" notes. Disambiguate the prior failed attempt at /workspace/@alkimiadev/alktype/ as "the @alkimiadev/alktype prototype" to distinguish it from this crate. Historical research citations (docs/research/*, /workspace/alknet-typedef-poc/) are kept as-is for provenance. Rename the crate in Cargo.toml ([package].name, [lib].name) and update the 11 use alknet_typedef::* imports across the 4 test files. Rebrand the crate-level doc comment in src/lib.rs. The TypeDef:* keyword strings, TypedefError/TypedefEngine identifiers, and other code-level references are unchanged — those are a separate code rebrand pass. Build, 295 tests, and clippy all pass clean.
This commit is contained in:
@@ -3,7 +3,7 @@ status: draft
|
||||
last_updated: 2026-07-22
|
||||
---
|
||||
|
||||
# alknet-typedef
|
||||
# alktype
|
||||
|
||||
The binary struct engine: a small Rust crate that takes a JSON Schema
|
||||
with `TypeDef:*` custom keywords and produces an offset map, read/write
|
||||
@@ -24,22 +24,22 @@ format definition; the engine is generic.
|
||||
|
||||
| ADR | Title | Relevance |
|
||||
|-----|-------|-----------|
|
||||
| [095](decisions/095-alknet-typedef-purpose-scope-jsonschema-engine.md) | Purpose, Scope, and the jsonschema Engine | What the crate is/isn't; why jsonschema not a custom engine; "schema is the format" principle; scope boundaries |
|
||||
| [096](decisions/096-two-layout-modes-packed-vs-aligned.md) | Two Layout Modes — Packed Sequential vs Aligned Static | The most important architectural finding; when to use each mode; `LayoutBuilder`/`SequentialReader` vs `OffsetMap` |
|
||||
| [097](decisions/097-schema-annotations.md) | Schema Annotations — Endianness, Alignment, Encoding, TUnion Discriminators | Concrete JSON shapes for all schema-level annotations |
|
||||
| [098](decisions/098-error-handling-validation-strategy.md) | Error Handling and Validation Strategy | `TypedefError` enum; load-time build, access-time check; field-path-carrying errors |
|
||||
| [099](decisions/099-int64-uint64-first-class-kinds.md) | Int64/Uint64 as First-Class Kinds | 64-bit integers (SFTP offsets, metatensor data_offsets); JSON precision caveat |
|
||||
| [100](decisions/100-reject-non-final-inline-length-prefixed-in-aligned-mode.md) | Reject Non-Final Inline Length-Prefixed Variable Fields in Aligned Mode | Prevents silent data corruption (inline variable data clobbering subsequent fields) |
|
||||
| [101](decisions/101-packed-mode-read-factory.md) | Packed-Mode Read API — Engine as SequentialReader Factory | `engine.sequential_reader()` returns an owned reader, not a reference |
|
||||
| [102](decisions/102-reject-tunion-in-aligned-mode.md) | Reject TUnion in Aligned Mode for v1 | Unions are the protocol pattern; aligned-mode union semantics were broken |
|
||||
| [001](decisions/001-alktype-purpose-scope-jsonschema-engine.md) | Purpose, Scope, and the jsonschema Engine | What the crate is/isn't; why jsonschema not a custom engine; "schema is the format" principle; scope boundaries |
|
||||
| [002](decisions/002-two-layout-modes-packed-vs-aligned.md) | Two Layout Modes — Packed Sequential vs Aligned Static | The most important architectural finding; when to use each mode; `LayoutBuilder`/`SequentialReader` vs `OffsetMap` |
|
||||
| [003](decisions/003-schema-annotations.md) | Schema Annotations — Endianness, Alignment, Encoding, TUnion Discriminators | Concrete JSON shapes for all schema-level annotations |
|
||||
| [004](decisions/004-error-handling-validation-strategy.md) | Error Handling and Validation Strategy | `TypedefError` enum; load-time build, access-time check; field-path-carrying errors |
|
||||
| [005](decisions/005-int64-uint64-first-class-kinds.md) | Int64/Uint64 as First-Class Kinds | 64-bit integers (SFTP offsets, metatensor data_offsets); JSON precision caveat |
|
||||
| [006](decisions/006-reject-non-final-inline-length-prefixed-in-aligned-mode.md) | Reject Non-Final Inline Length-Prefixed Variable Fields in Aligned Mode | Prevents silent data corruption (inline variable data clobbering subsequent fields) |
|
||||
| [007](decisions/007-packed-mode-read-factory.md) | Packed-Mode Read API — Engine as SequentialReader Factory | `engine.sequential_reader()` returns an owned reader, not a reference |
|
||||
| [008](decisions/008-reject-tunion-in-aligned-mode.md) | Reject TUnion in Aligned Mode for v1 | Unions are the protocol pattern; aligned-mode union semantics were broken |
|
||||
|
||||
## Relevant Open Questions
|
||||
|
||||
| OQ | Title | Status | Relevance |
|
||||
|----|-------|--------|-----------|
|
||||
| OQ-069 | Arrays of variable-length-element structs | deferred(scope) | Requires lazy walking logic; blocked on a concrete consumer that needs it |
|
||||
| OQ-070 | `no_std` + `alloc` support | deferred(scope) | Target `std` for v1; blocked on an embedded use case |
|
||||
| OQ-071 | Builder API for schema construction | deferred(scope) | Schemas are authored in TypeBox or hand-written JSON for v1; blocked on a concrete need |
|
||||
| OQ-001 | Arrays of variable-length-element structs | deferred(scope) | Requires lazy walking logic; blocked on a concrete consumer that needs it |
|
||||
| OQ-002 | `no_std` + `alloc` support | deferred(scope) | Target `std` for v1; blocked on an embedded use case |
|
||||
| OQ-003 | Builder API for schema construction | deferred(scope) | Schemas are authored in TypeBox or hand-written JSON for v1; blocked on a concrete need |
|
||||
|
||||
## Key Design Principles
|
||||
|
||||
@@ -47,53 +47,53 @@ format definition; the engine is generic.
|
||||
keywords is both the validation spec and the layout spec. No separate
|
||||
format definition, no separate parser, no separate validator. One
|
||||
schema, three uses: validate, compute offsets, access data. See
|
||||
[overview.md](overview.md) and [ADR-095](decisions/095-alknet-typedef-purpose-scope-jsonschema-engine.md).
|
||||
[overview.md](overview.md) and [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md).
|
||||
|
||||
2. **jsonschema is the validation engine, not a custom engine.** The
|
||||
`jsonschema` crate (v0.46.5, Draft 2020-12) handles validation with
|
||||
custom keyword support. The novel code is the offset computation, not
|
||||
the validation. This eliminates ~14,000 lines of hand-rolled schema
|
||||
engines (typebox-rs, alktype). See [schema-layer.md](schema-layer.md)
|
||||
and [ADR-095](decisions/095-alknet-typedef-purpose-scope-jsonschema-engine.md).
|
||||
engines (typebox-rs, the @alkimiadev/alktype prototype). See [schema-layer.md](schema-layer.md)
|
||||
and [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md).
|
||||
|
||||
3. **Two layout modes for two use cases.** Packed sequential
|
||||
(`LayoutBuilder`/`SequentialReader`) for protocol wire formats (SFTP,
|
||||
channels, TTY). Aligned static (`OffsetMap`) for mmap-friendly formats
|
||||
(metatensor). The consumer selects the mode; the schema is the same.
|
||||
See [layout-engine.md](layout-engine.md) and
|
||||
[ADR-096](decisions/096-two-layout-modes-packed-vs-aligned.md).
|
||||
[ADR-002](decisions/002-two-layout-modes-packed-vs-aligned.md).
|
||||
|
||||
4. **Variable-length types default to inline length-prefixing.**
|
||||
`[length: u32][data]` is the universal pattern used by channels, SFTP,
|
||||
TTY, and most binary protocols. Offset indirection (the metatensor
|
||||
blob tensor pattern) is opt-in via the `encoding` annotation. See
|
||||
[layout-engine.md](layout-engine.md) and
|
||||
[ADR-097](decisions/097-schema-annotations.md).
|
||||
[ADR-003](decisions/003-schema-annotations.md).
|
||||
|
||||
5. **TUnion supports both byte-offset and field-name discriminators.**
|
||||
Byte-offset for protocol dispatch (SFTP type bytes, call protocol
|
||||
event types). Field-name for the typedef.ts string pattern. See
|
||||
[data-access.md](data-access.md) and
|
||||
[ADR-097](decisions/097-schema-annotations.md).
|
||||
[ADR-003](decisions/003-schema-annotations.md).
|
||||
|
||||
6. **Endianness is per-schema, default little-endian.** The engine reads
|
||||
the `"endian"` annotation and byte-swaps accordingly. SFTP consumers
|
||||
specify `"endian": "big"`. See [layout-engine.md](layout-engine.md)
|
||||
and [ADR-097](decisions/097-schema-annotations.md).
|
||||
and [ADR-003](decisions/003-schema-annotations.md).
|
||||
|
||||
7. **Validation is opt-in, built once at load time.** The jsonschema
|
||||
validator is compiled once at schema load time. Access-time validation
|
||||
is a fast `is_valid()` check. High-throughput paths can skip
|
||||
validation; security-sensitive paths can validate every frame. See
|
||||
[validation.md](validation.md) and
|
||||
[ADR-098](decisions/098-error-handling-validation-strategy.md).
|
||||
[ADR-004](decisions/004-error-handling-validation-strategy.md).
|
||||
|
||||
8. **Not a serialization framework.** The typedef engine is not a
|
||||
general-purpose serde replacement. It operates on raw byte buffers at
|
||||
computed offsets — no intermediate `Value` tree, no reflection, no
|
||||
dynamic dispatch per field. For JSON data, use serde. For binary data
|
||||
with a known schema, use typedef. See [overview.md](overview.md) and
|
||||
[ADR-095](decisions/095-alknet-typedef-purpose-scope-jsonschema-engine.md).
|
||||
[ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md).
|
||||
|
||||
## References
|
||||
|
||||
@@ -107,4 +107,4 @@ format definition; the engine is generic.
|
||||
- `/workspace/jsonschema/` — the jsonschema crate (v0.46.5, Draft 2020-12)
|
||||
- `/workspace/alknet-typedef-poc/` — the POC code (disposable)
|
||||
- `/workspace/@alkimiadev/typebox-rs/` — prior attempt, replaced by typedef
|
||||
- `/workspace/@alkimiadev/alktype/` — prior attempt, replaced by typedef
|
||||
- `/workspace/@alkimiadev/alktype/` — prior attempt (the @alkimiadev/alktype prototype, a handler-registry pattern; not to be confused with this crate, which reuses the name but is backed by the `jsonschema` crate)
|
||||
|
||||
@@ -3,7 +3,7 @@ status: draft
|
||||
last_updated: 2026-07-22
|
||||
---
|
||||
|
||||
# alknet-typedef — Data Access
|
||||
# alktype — Data Access
|
||||
|
||||
The data access layer: read/write functions, TUnion dispatch, field paths,
|
||||
zero-copy access for fixed-size types, and length-prefix reading for
|
||||
@@ -79,7 +79,7 @@ impl TypedefEngine {
|
||||
pub fn write_field(&self, buffer: &mut [u8], field_path: &str,
|
||||
value: &FieldValue<'_>) -> Result<(), TypedefError>;
|
||||
|
||||
// Packed mode: returns an owned fresh SequentialReader (ADR-101).
|
||||
// Packed mode: returns an owned fresh SequentialReader (ADR-007).
|
||||
// Each call returns a new reader with the cursor at position 0.
|
||||
// The consumer owns the reader and drives read_next/read_field/reset.
|
||||
pub fn sequential_reader(&self) -> Option<SequentialReader>;
|
||||
@@ -146,7 +146,7 @@ fn write_u32(buffer: &mut [u8], offset: usize, value: u32,
|
||||
```
|
||||
|
||||
The engine applies endianness at access time based on the schema's
|
||||
`"endian"` annotation (ADR-097). The offset computation is
|
||||
`"endian"` annotation (ADR-003). The offset computation is
|
||||
endian-agnostic. The `read_array`/`write_array` helpers perform the
|
||||
bounds check and produce `TypedefError::Access` with the field path on
|
||||
failure.
|
||||
@@ -357,22 +357,22 @@ engine returns a slice of that region.
|
||||
## Error Handling
|
||||
|
||||
Read/write errors carry the field path for debugging. See
|
||||
[ADR-098](decisions/098-error-handling-validation-strategy.md) and
|
||||
[ADR-004](decisions/004-error-handling-validation-strategy.md) and
|
||||
[validation.md](validation.md) for the full error model.
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| Decision | ADR | Summary |
|
||||
|----------|-----|---------|
|
||||
| Two layout modes | [ADR-096](decisions/096-two-layout-modes-packed-vs-aligned.md) | Determines whether offsets are fixed (OffsetMap) or sequential (SequentialReader) |
|
||||
| Schema annotations | [ADR-097](decisions/097-schema-annotations.md) | Endianness, encoding, and TUnion discriminator shapes that control data access |
|
||||
| Error handling | [ADR-098](decisions/098-error-handling-validation-strategy.md) | Field-path-carrying errors for read/write operations |
|
||||
| Two layout modes | [ADR-002](decisions/002-two-layout-modes-packed-vs-aligned.md) | Determines whether offsets are fixed (OffsetMap) or sequential (SequentialReader) |
|
||||
| Schema annotations | [ADR-003](decisions/003-schema-annotations.md) | Endianness, encoding, and TUnion discriminator shapes that control data access |
|
||||
| Error handling | [ADR-004](decisions/004-error-handling-validation-strategy.md) | Field-path-carrying errors for read/write operations |
|
||||
|
||||
## Open Questions
|
||||
|
||||
See [open-questions.md](open-questions.md) for full details.
|
||||
|
||||
- **OQ-069** (deferred(scope)): Arrays of variable-length-element structs
|
||||
- **OQ-001** (deferred(scope)): Arrays of variable-length-element structs
|
||||
— affects the sequential walking logic for array access.
|
||||
|
||||
## References
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# ADR-095: alknet-typedef — Purpose, Scope, and the jsonschema Engine
|
||||
# ADR-001: alktype — Purpose, Scope, and the jsonschema Engine
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
@@ -37,9 +37,11 @@ Two prior attempts built their own jsonschema engines — the fatal flaw:
|
||||
- **typebox-rs** (`/workspace/@alkimiadev/typebox-rs/`, ~8,400 lines):
|
||||
a full 26-variant `SchemaKind` enum, a custom `Value` type with typed
|
||||
arrays, and a 912-line hand-written validator.
|
||||
- **alktype** (`/workspace/@alkimiadev/alktype/`, ~5,600 lines): a
|
||||
handler-registry pattern that also implements its own validation for
|
||||
each type.
|
||||
- **the @alkimiadev/alktype prototype** (`/workspace/@alkimiadev/alktype/`,
|
||||
~5,600 lines): a handler-registry pattern that also implements its own
|
||||
validation for each type. (Not to be confused with this crate, which
|
||||
reuses the name but is a fresh implementation backed by the `jsonschema`
|
||||
crate rather than a hand-rolled engine.)
|
||||
|
||||
The `jsonschema` crate (v0.46.5, Draft 2020-12) is already in the
|
||||
workspace at `/workspace/jsonschema/`. It handles validation with custom
|
||||
@@ -57,7 +59,7 @@ functions, and validation — all driven by the schema.
|
||||
|
||||
## Decision
|
||||
|
||||
**alknet-typedef is a small Rust crate that takes a JSON Schema with
|
||||
**alktype is a small Rust crate that takes a JSON Schema with
|
||||
`TypeDef:*` custom keywords and produces three capabilities:**
|
||||
|
||||
1. **An offset map** — walks the schema, computes byte offsets for each
|
||||
@@ -94,16 +96,16 @@ JSON Schema wire format.
|
||||
**The crate targets `std` for v1.** The WASM target has `std` available
|
||||
via `wasm-bindgen`. If embedded use cases emerge, `no_std` + `alloc` can
|
||||
be added as a feature gate later — the engine's core (offset computation,
|
||||
read/write) is already allocation-free. See OQ-070.
|
||||
read/write) is already allocation-free. See OQ-002.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- **Eliminates ~14,000 lines of hand-rolled schema engines.** typebox-rs
|
||||
and alktype are replaced by `jsonschema` + an offset map + ~50 lines of
|
||||
custom keyword implementations. The codebase drops from "a port of
|
||||
TypeBox" to "jsonschema + an offset map."
|
||||
and the @alkimiadev/alktype prototype are replaced by `jsonschema` + an
|
||||
offset map + ~50 lines of custom keyword implementations. The codebase
|
||||
drops from "a port of TypeBox" to "jsonschema + an offset map."
|
||||
- **One schema, three uses.** The same JSON Schema validates, computes
|
||||
offsets, and drives data access. No separate format definition, parser,
|
||||
or validator per protocol.
|
||||
@@ -134,7 +136,7 @@ read/write) is already allocation-free. See OQ-070.
|
||||
small compile-time cost.
|
||||
- **Schema authoring is external.** Schemas are authored in TypeBox (JS)
|
||||
or hand-written JSON. The typedef engine consumes schemas; it does not
|
||||
generate them. A builder API is deferred (OQ-071).
|
||||
generate them. A builder API is deferred (OQ-003).
|
||||
|
||||
## Scope Boundaries (What This Is Not)
|
||||
|
||||
@@ -166,8 +168,8 @@ read/write) is already allocation-free. See OQ-070.
|
||||
schema kinds (619 lines)
|
||||
- `/workspace/jsonschema/` — the jsonschema crate (v0.46.5, Draft 2020-12)
|
||||
- `/workspace/alknet-typedef-poc/` — the POC code (disposable)
|
||||
- [ADR-096](096-two-layout-modes-packed-vs-aligned.md) — the two layout
|
||||
- [ADR-002](002-two-layout-modes-packed-vs-aligned.md) — the two layout
|
||||
modes decision
|
||||
- [ADR-097](097-schema-annotations.md) — schema annotation shapes
|
||||
- [ADR-098](098-error-handling-validation-strategy.md) — error handling
|
||||
- [ADR-003](003-schema-annotations.md) — schema annotation shapes
|
||||
- [ADR-004](004-error-handling-validation-strategy.md) — error handling
|
||||
and validation strategy
|
||||
@@ -1,4 +1,4 @@
|
||||
# ADR-096: Two Layout Modes — Packed Sequential vs Aligned Static
|
||||
# ADR-002: Two Layout Modes — Packed Sequential vs Aligned Static
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
@@ -95,7 +95,7 @@ fields).
|
||||
Inline length-prefixing (`[length: u32][data]`) is the default for all
|
||||
variable-length types in both modes. This is the universal pattern used
|
||||
by channels, SFTP, TTY, and most binary protocols. Offset indirection is
|
||||
opt-in via the `encoding` annotation (see ADR-097).
|
||||
opt-in via the `encoding` annotation (see ADR-003).
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -131,7 +131,7 @@ opt-in via the `encoding` annotation (see ADR-097).
|
||||
|
||||
- `docs/research/alknet-typedef/findings.md` §"POC Results" — POC 1
|
||||
(aligned OffsetMap) and POC 2 (packed LayoutBuilder/SequentialReader)
|
||||
- [ADR-095](095-alknet-typedef-purpose-scope-jsonschema-engine.md) —
|
||||
- [ADR-001](001-alktype-purpose-scope-jsonschema-engine.md) —
|
||||
purpose and scope
|
||||
- [ADR-097](097-schema-annotations.md) — schema annotations including
|
||||
- [ADR-003](003-schema-annotations.md) — schema annotations including
|
||||
the `encoding` field for variable-length types
|
||||
@@ -1,4 +1,4 @@
|
||||
# ADR-097: Schema Annotations — Endianness, Alignment, Encoding, and TUnion Discriminators
|
||||
# ADR-003: Schema Annotations — Endianness, Alignment, Encoding, and TUnion Discriminators
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
@@ -65,7 +65,7 @@ struct-level.**
|
||||
- Default alignment (when no annotation is present): 1 for u8/bool, 2
|
||||
for u16/i16, 4 for u32/i32/f32, 8 for u64/i64/f64, max field alignment
|
||||
for structs.
|
||||
- Alignment is only meaningful in aligned static mode (ADR-096). In
|
||||
- Alignment is only meaningful in aligned static mode (ADR-002). In
|
||||
packed sequential mode, alignment annotations are ignored — fields are
|
||||
packed with no padding.
|
||||
|
||||
@@ -253,7 +253,7 @@ event types). Both work.
|
||||
|
||||
- `docs/research/alknet-typedef/findings.md` §"Open Questions" — the
|
||||
annotation shape questions this ADR resolves
|
||||
- [ADR-095](095-alknet-typedef-purpose-scope-jsonschema-engine.md) —
|
||||
- [ADR-001](001-alktype-purpose-scope-jsonschema-engine.md) —
|
||||
purpose and scope
|
||||
- [ADR-096](096-two-layout-modes-packed-vs-aligned.md) — the two layout
|
||||
- [ADR-002](002-two-layout-modes-packed-vs-aligned.md) — the two layout
|
||||
modes (alignment only meaningful in aligned static mode)
|
||||
@@ -1,4 +1,4 @@
|
||||
# ADR-098: Error Handling and Validation Strategy
|
||||
# ADR-004: Error Handling and Validation Strategy
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
@@ -150,8 +150,8 @@ you exactly which field failed and why.
|
||||
|
||||
- `docs/research/alknet-typedef/findings.md` §"Open Questions" — error
|
||||
handling strategy question (OQ 8)
|
||||
- [ADR-095](095-alknet-typedef-purpose-scope-jsonschema-engine.md) —
|
||||
- [ADR-001](001-alktype-purpose-scope-jsonschema-engine.md) —
|
||||
purpose and scope
|
||||
- [ADR-096](096-two-layout-modes-packed-vs-aligned.md) — the two layout
|
||||
- [ADR-002](002-two-layout-modes-packed-vs-aligned.md) — the two layout
|
||||
modes
|
||||
- [ADR-097](097-schema-annotations.md) — schema annotations
|
||||
- [ADR-003](003-schema-annotations.md) — schema annotations
|
||||
@@ -1,11 +1,11 @@
|
||||
# ADR-099: Int64/Uint64 as First-Class Kinds
|
||||
# ADR-005: Int64/Uint64 as First-Class Kinds
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The typedef engine's kind set (ADR-095, ADR-097) tops out at 32-bit
|
||||
The typedef engine's kind set (ADR-001, ADR-003) tops out at 32-bit
|
||||
integers. The POC included `u64` read/write primitives, and the
|
||||
call-channels-unification research's own SFTP schema example uses
|
||||
`"TypeDef:Uint64"` for the `offset` field (`Read`/`Write` packets have
|
||||
@@ -108,7 +108,7 @@ The engine now has **19** first-class kinds (17 + Int64 + Uint64).
|
||||
the SFTP schema with `"offset": { "TypeDef:Uint64": true }`
|
||||
- `docs/research/alknet-typedef/findings.md` §"POC 1" — the POC included
|
||||
u64 read/write
|
||||
- [ADR-095](095-alknet-typedef-purpose-scope-jsonschema-engine.md) —
|
||||
- [ADR-001](001-alktype-purpose-scope-jsonschema-engine.md) —
|
||||
purpose and scope (the kind set)
|
||||
- [ADR-097](097-schema-annotations.md) — schema annotations
|
||||
- [ADR-003](003-schema-annotations.md) — schema annotations
|
||||
(endianness applies to the new kinds)
|
||||
@@ -1,11 +1,11 @@
|
||||
# ADR-100: Reject Non-Final Inline Length-Prefixed Variable Fields in Aligned Mode
|
||||
# ADR-006: Reject Non-Final Inline Length-Prefixed Variable Fields in Aligned Mode
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The aligned static layout mode (ADR-096) is designed for mmap-friendly
|
||||
The aligned static layout mode (ADR-002) is designed for mmap-friendly
|
||||
formats: fields have fixed positions with natural alignment padding,
|
||||
enabling random access by field path without parsing preceding fields.
|
||||
|
||||
@@ -102,9 +102,9 @@ not when writing data.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-096](096-two-layout-modes-packed-vs-aligned.md) — the two layout
|
||||
- [ADR-002](002-two-layout-modes-packed-vs-aligned.md) — the two layout
|
||||
modes (aligned static mode's variable-length handling)
|
||||
- [ADR-097](097-schema-annotations.md) — the three variable-length
|
||||
- [ADR-003](003-schema-annotations.md) — the three variable-length
|
||||
encoding strategies (`maxLength`, `offset-indirect`, inline
|
||||
length-prefixing)
|
||||
- `../layout-engine.md` §"Variable-length
|
||||
@@ -1,4 +1,4 @@
|
||||
# ADR-101: Packed-Mode Read API — Engine as SequentialReader Factory
|
||||
# ADR-007: Packed-Mode Read API — Engine as SequentialReader Factory
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
@@ -97,7 +97,7 @@ compared to the cost of reading a buffer.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-096](096-two-layout-modes-packed-vs-aligned.md) — packed
|
||||
- [ADR-002](002-two-layout-modes-packed-vs-aligned.md) — packed
|
||||
sequential mode (`SequentialReader` as the read-side)
|
||||
- `../data-access.md` §"Higher-level
|
||||
read/write" — the `SequentialReader` API
|
||||
@@ -1,11 +1,11 @@
|
||||
# ADR-102: Reject TUnion in Aligned Mode for v1
|
||||
# ADR-008: Reject TUnion in Aligned Mode for v1
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The aligned static layout mode (ADR-096) computes fixed byte positions
|
||||
The aligned static layout mode (ADR-002) computes fixed byte positions
|
||||
for each field, enabling random access by field path. `TUnion` in
|
||||
aligned mode has three implementation problems:
|
||||
|
||||
@@ -104,8 +104,8 @@ arrives, not speculatively now.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-096](096-two-layout-modes-packed-vs-aligned.md) — the two layout
|
||||
- [ADR-002](002-two-layout-modes-packed-vs-aligned.md) — the two layout
|
||||
modes
|
||||
- [ADR-097](097-schema-annotations.md) §4 — TUnion discriminators
|
||||
- [ADR-003](003-schema-annotations.md) §4 — TUnion discriminators
|
||||
- `../layout-engine.md` §"TUnion" —
|
||||
aligned-mode union sizing
|
||||
@@ -3,7 +3,7 @@ status: draft
|
||||
last_updated: 2026-07-22
|
||||
---
|
||||
|
||||
# alknet-typedef — Layout Engine
|
||||
# alktype — Layout Engine
|
||||
|
||||
The layout engine: offset computation, the two layout modes (packed
|
||||
sequential vs aligned static), alignment, endianness, and variable-length
|
||||
@@ -14,7 +14,7 @@ JSON that computes byte positions for each field.
|
||||
|
||||
The POCs surfaced that protocols and mmap-friendly formats need different
|
||||
layout strategies. This is the most important architectural finding —
|
||||
decided in [ADR-096](decisions/096-two-layout-modes-packed-vs-aligned.md).
|
||||
decided in [ADR-002](decisions/002-two-layout-modes-packed-vs-aligned.md).
|
||||
|
||||
### Mode 1: Packed sequential (protocol wire formats)
|
||||
|
||||
@@ -114,7 +114,7 @@ with a `TypedefError::Offset` — the `OffsetMap` reserves only 4 bytes
|
||||
data inline, which would clobber subsequent fields. Non-final variable
|
||||
fields must use `maxLength` (fixed-size reservation) or
|
||||
`"encoding": "offset-indirect"`. See
|
||||
[ADR-100](decisions/100-reject-non-final-inline-length-prefixed-in-aligned-mode.md).
|
||||
[ADR-006](decisions/006-reject-non-final-inline-length-prefixed-in-aligned-mode.md).
|
||||
|
||||
## Offset Computation Algorithm
|
||||
|
||||
@@ -142,7 +142,7 @@ its total size.
|
||||
**`TUnion`:** TUnion is supported in packed sequential mode only. In
|
||||
aligned static mode, `OffsetMap::compute` rejects `TUnion` fields with
|
||||
`TypedefError::Offset` — see
|
||||
[ADR-102](decisions/102-reject-tunion-in-aligned-mode.md). Unions
|
||||
[ADR-008](decisions/008-reject-tunion-in-aligned-mode.md). Unions
|
||||
are the protocol dispatch pattern (SFTP type bytes, call protocol event
|
||||
types); mmap-friendly formats use structs and arrays, not tagged unions.
|
||||
|
||||
@@ -166,13 +166,13 @@ alignment padding in aligned mode). Element `i` starts at
|
||||
`array_offset + i × stride`. The array's total size is `count × stride`.
|
||||
|
||||
**`TArray` of variable-length-element structs:** Deferred for v1
|
||||
(OQ-069).
|
||||
(OQ-001).
|
||||
|
||||
### Variable-length types
|
||||
|
||||
The typedef engine supports three strategies for variable-length types
|
||||
(see [schema-layer.md](schema-layer.md) §Variable-length types and
|
||||
[ADR-097](decisions/097-schema-annotations.md) §3 for the full
|
||||
[ADR-003](decisions/003-schema-annotations.md) §3 for the full
|
||||
annotation shapes).
|
||||
|
||||
**Strategy 1: Inline length-prefixing (default).**
|
||||
@@ -211,7 +211,7 @@ path prefix.
|
||||
|
||||
### Endianness
|
||||
|
||||
Endianness is per-schema (ADR-097). The offset computation is
|
||||
Endianness is per-schema (ADR-003). The offset computation is
|
||||
endian-agnostic — it computes byte positions, not byte values. The
|
||||
read/write functions apply endianness when converting between bytes and
|
||||
typed values. The engine reads the `"endian"` annotation from the schema
|
||||
@@ -252,7 +252,7 @@ returns `Some(&OffsetMap)` in aligned mode and `None` in packed mode;
|
||||
and `None` in aligned mode. `engine.sequential_reader()` returns
|
||||
`Option<SequentialReader>` (an owned fresh reader, not a reference — the
|
||||
reader has mutable cursor state that the consumer owns; see
|
||||
[ADR-101](decisions/101-packed-mode-read-factory.md)) in packed
|
||||
[ADR-007](decisions/007-packed-mode-read-factory.md)) in packed
|
||||
mode and `None` in aligned mode. See [validation.md](validation.md)
|
||||
§"The TypedefEngine struct" for the engine API.
|
||||
|
||||
@@ -332,17 +332,17 @@ order (schema `properties` order, nested struct fields appearing inline).
|
||||
|
||||
| Decision | ADR | Summary |
|
||||
|----------|-----|---------|
|
||||
| Two layout modes | [ADR-096](decisions/096-two-layout-modes-packed-vs-aligned.md) | Packed sequential for protocols; aligned static for mmap formats |
|
||||
| Schema annotations | [ADR-097](decisions/097-schema-annotations.md) | Endianness, alignment, encoding annotations that control layout behavior |
|
||||
| Non-final inline variable fields | [ADR-100](decisions/100-reject-non-final-inline-length-prefixed-in-aligned-mode.md) | Rejected in aligned mode (would clobber subsequent fields); use `maxLength` or `offset-indirect` |
|
||||
| Packed-mode read factory | [ADR-101](decisions/101-packed-mode-read-factory.md) | `engine.sequential_reader()` returns an owned fresh reader, not a reference |
|
||||
| TUnion in aligned mode | [ADR-102](decisions/102-reject-tunion-in-aligned-mode.md) | Rejected for v1 (broken semantics; no current consumer needs it) |
|
||||
| Two layout modes | [ADR-002](decisions/002-two-layout-modes-packed-vs-aligned.md) | Packed sequential for protocols; aligned static for mmap formats |
|
||||
| Schema annotations | [ADR-003](decisions/003-schema-annotations.md) | Endianness, alignment, encoding annotations that control layout behavior |
|
||||
| Non-final inline variable fields | [ADR-006](decisions/006-reject-non-final-inline-length-prefixed-in-aligned-mode.md) | Rejected in aligned mode (would clobber subsequent fields); use `maxLength` or `offset-indirect` |
|
||||
| Packed-mode read factory | [ADR-007](decisions/007-packed-mode-read-factory.md) | `engine.sequential_reader()` returns an owned fresh reader, not a reference |
|
||||
| TUnion in aligned mode | [ADR-008](decisions/008-reject-tunion-in-aligned-mode.md) | Rejected for v1 (broken semantics; no current consumer needs it) |
|
||||
|
||||
## Open Questions
|
||||
|
||||
See [open-questions.md](open-questions.md) for full details.
|
||||
|
||||
- **OQ-069** (deferred(scope)): Arrays of variable-length-element structs
|
||||
- **OQ-001** (deferred(scope)): Arrays of variable-length-element structs
|
||||
— requires lazy walking logic; blocked on a concrete consumer that
|
||||
needs it.
|
||||
|
||||
@@ -350,9 +350,9 @@ See [open-questions.md](open-questions.md) for full details.
|
||||
|
||||
- `docs/research/alknet-typedef/findings.md` §"POC Results" — POC 1
|
||||
(aligned OffsetMap) and POC 2 (packed LayoutBuilder/SequentialReader)
|
||||
- [ADR-096](decisions/096-two-layout-modes-packed-vs-aligned.md) —
|
||||
- [ADR-002](decisions/002-two-layout-modes-packed-vs-aligned.md) —
|
||||
the two layout modes decision
|
||||
- [ADR-097](decisions/097-schema-annotations.md) — schema
|
||||
- [ADR-003](decisions/003-schema-annotations.md) — schema
|
||||
annotations
|
||||
- [schema-layer.md](schema-layer.md) — the 17 TypeDef kinds and their
|
||||
byte sizes
|
||||
|
||||
@@ -35,7 +35,7 @@ hub deployment because the hub dials workers" not "blocks the hub
|
||||
crate." This is the triage signal that makes the deferral's urgency
|
||||
visible.
|
||||
|
||||
Door type classifications follow ADR-009 — they describe **reversal cost** (how expensive it is to undo), not urgency:
|
||||
Door type classifications describe **reversal cost** (how expensive it is to undo), not urgency:
|
||||
- **One-way door**: Reversal requires rewriting significant code or permanently closes a capability. Getting it wrong is expensive — requires ADR before implementation.
|
||||
- **Two-way door**: Reversal is cheap or additive. Getting it wrong is recoverable — decide, implement, revert if needed.
|
||||
|
||||
@@ -47,19 +47,19 @@ Door type is separate from whether a decision is made. A two-way door is a decis
|
||||
|
||||
| OQ | Title | Status | Door | Pri |
|
||||
|----|-------|--------|------|-----|
|
||||
| [OQ-069](questions/069-arrays-of-variable-length-element-structs.md) | Arrays of Variable-Length-Element Structs | deferred(scope) | two | low |
|
||||
| [OQ-001](questions/001-arrays-of-variable-length-element-structs.md) | Arrays of Variable-Length-Element Structs | deferred(scope) | two | low |
|
||||
|
||||
### Platform Support
|
||||
|
||||
| OQ | Title | Status | Door | Pri |
|
||||
|----|-------|--------|------|-----|
|
||||
| [OQ-070](questions/070-no-std-alloc-support.md) | `no_std` + `alloc` Support | deferred(scope) | two | low |
|
||||
| [OQ-002](questions/002-no-std-alloc-support.md) | `no_std` + `alloc` Support | deferred(scope) | two | low |
|
||||
|
||||
### Schema Construction
|
||||
|
||||
| OQ | Title | Status | Door | Pri |
|
||||
|----|-------|--------|------|-----|
|
||||
| [OQ-071](questions/071-builder-api-for-schema-construction.md) | Builder API for Schema Construction | deferred(scope) | two | med |
|
||||
| [OQ-003](questions/003-builder-api-for-schema-construction.md) | Builder API for Schema Construction | deferred(scope) | two | med |
|
||||
|
||||
## Deferred / Blocked
|
||||
|
||||
@@ -69,7 +69,7 @@ blocking condition. They are not failures; they are scope management.
|
||||
This section exists so "what's currently blocking the architect" is
|
||||
answerable at a glance, not by filtering the tables above.
|
||||
|
||||
### OQ-069: Arrays of Variable-Length-Element Structs
|
||||
### OQ-001: Arrays of Variable-Length-Element Structs
|
||||
|
||||
- **Blocked on**: A concrete consumer that needs arrays of structs with
|
||||
variable-length fields, where the elements are interleaved
|
||||
@@ -80,9 +80,9 @@ answerable at a glance, not by filtering the tables above.
|
||||
pattern), not as an array of fixed-stride structs. Arrays of
|
||||
fixed-size structs are fully supported.
|
||||
- **Priority**: low
|
||||
- **Full file**: [OQ-069](questions/069-arrays-of-variable-length-element-structs.md)
|
||||
- **Full file**: [OQ-001](questions/001-arrays-of-variable-length-element-structs.md)
|
||||
|
||||
### OQ-070: `no_std` + `alloc` Support
|
||||
### OQ-002: `no_std` + `alloc` Support
|
||||
|
||||
- **Blocked on**: An embedded use case that requires `no_std` + `alloc`
|
||||
(e.g., a microcontroller running Rust without `std`). The WASM target
|
||||
@@ -90,9 +90,9 @@ answerable at a glance, not by filtering the tables above.
|
||||
computation, read/write) is already allocation-free; the `jsonschema`
|
||||
dependency is the only `alloc` consumer.
|
||||
- **Priority**: low
|
||||
- **Full file**: [OQ-070](questions/070-no-std-alloc-support.md)
|
||||
- **Full file**: [OQ-002](questions/002-no-std-alloc-support.md)
|
||||
|
||||
### OQ-071: Builder API for Schema Construction
|
||||
### OQ-003: Builder API for Schema Construction
|
||||
|
||||
- **Blocked on**: A concrete need for programmatic schema construction
|
||||
in Rust. The current consumers (SFTP, metatensor, binary call frames,
|
||||
@@ -101,4 +101,4 @@ answerable at a glance, not by filtering the tables above.
|
||||
produces the same JSON Schema structure — it would sit on top of the
|
||||
engine, not inside it.
|
||||
- **Priority**: medium
|
||||
- **Full file**: [OQ-071](questions/071-builder-api-for-schema-construction.md)
|
||||
- **Full file**: [OQ-003](questions/003-builder-api-for-schema-construction.md)
|
||||
@@ -3,7 +3,7 @@ status: draft
|
||||
last_updated: 2026-07-22
|
||||
---
|
||||
|
||||
# alknet-typedef — Overview
|
||||
# alktype — Overview
|
||||
|
||||
The binary struct engine: a small Rust crate that takes a JSON Schema
|
||||
with `TypeDef:*` custom keywords and produces an offset map, read/write
|
||||
@@ -16,10 +16,10 @@ Component details are in the sibling documents.
|
||||
|
||||
## What
|
||||
|
||||
`alknet-typedef` is a library crate that consumes JSON Schemas annotated
|
||||
`alktype` is a library crate that consumes JSON Schemas annotated
|
||||
with `TypeDef:*` custom keywords (the same kinds defined in TypeBox's
|
||||
`typedef.ts`, plus `TypeDef:Bytes`, `TypeDef:Int64`, and `TypeDef:Uint64`
|
||||
as alknet-typedef additions) and produces three capabilities:
|
||||
as alktype additions) and produces three capabilities:
|
||||
|
||||
1. **An offset map** — walks the schema, computes byte offsets for each
|
||||
field based on type sizes, field order, and alignment.
|
||||
@@ -36,18 +36,19 @@ each field. The custom keyword implementations are small (a few lines
|
||||
each, generated from shared macros — see [validation.md](validation.md)).
|
||||
|
||||
The crate replaces two prior attempts that built their own jsonschema
|
||||
engines — typebox-rs (~8,400 lines) and alktype (~5,600 lines) — with
|
||||
`jsonschema` + an offset map + small custom keyword implementations. See
|
||||
[ADR-095](decisions/095-alknet-typedef-purpose-scope-jsonschema-engine.md).
|
||||
engines — typebox-rs (~8,400 lines) and the @alkimiadev/alktype prototype
|
||||
(~5,600 lines) — with `jsonschema` + an offset map + small custom keyword
|
||||
implementations. See
|
||||
[ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md).
|
||||
|
||||
## Why
|
||||
|
||||
The crate's purpose is to be the binary struct engine for every alknet
|
||||
component that reads or writes binary data at computed offsets. Instead
|
||||
of per-protocol serde structs (russh-sftp's 29 packet types), per-handler
|
||||
wire format code (TTY's 5-byte format parser), or per-format offset
|
||||
computation (metatensor's tensor access), all of these become instances
|
||||
of the same engine with different schemas.
|
||||
The crate's purpose is to be a binary struct engine for components that
|
||||
read or write binary data at computed offsets. Instead of per-protocol
|
||||
serde structs (russh-sftp's 29 packet types), per-handler wire format
|
||||
code (TTY's 5-byte format parser), or per-format offset computation
|
||||
(metatensor's tensor access), all of these become instances of the same
|
||||
engine with different schemas.
|
||||
|
||||
The guiding insight:
|
||||
|
||||
@@ -95,17 +96,16 @@ language-specific annotations. The schema is the ABI contract.
|
||||
## Dependencies
|
||||
|
||||
```
|
||||
alknet-typedef
|
||||
alktype
|
||||
├── jsonschema (v0.46.5, Draft 2020-12) — validation engine, custom keyword support
|
||||
├── serde_json (with preserve_order) — schema parsing; field order is load-bearing
|
||||
└── (no tokio, no platform deps) — WASM-clean by construction
|
||||
```
|
||||
|
||||
`alknet-typedef` is dependency-light: `jsonschema` + `serde_json` only.
|
||||
`alktype` is dependency-light: `jsonschema` + `serde_json` only.
|
||||
No tokio, no platform deps. Compiles to `wasm32-unknown-unknown` for
|
||||
browser use. The `jsonschema` crate is already in the workspace at
|
||||
`/workspace/jsonschema/` but not yet used by any alknet crate — typedef
|
||||
is the first consumer.
|
||||
`/workspace/jsonschema/` — alktype is its first consumer.
|
||||
|
||||
`serde_json` requires the `preserve_order` feature because field order
|
||||
is load-bearing for binary layouts. The order of properties in the
|
||||
@@ -131,7 +131,7 @@ offsets, reads fields. Same result, no per-packet-type code.
|
||||
|
||||
## Scope Boundaries (What This Is Not)
|
||||
|
||||
These boundaries are decided in [ADR-095](decisions/095-alknet-typedef-purpose-scope-jsonschema-engine.md).
|
||||
These boundaries are decided in [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md).
|
||||
|
||||
- **Not metatensor.** typedef is the binary struct *engine*. Metatensor
|
||||
is a *format* (8-byte header + JSON header + binary data) that uses the
|
||||
@@ -145,7 +145,7 @@ These boundaries are decided in [ADR-095](decisions/095-alknet-typedef-purpose-s
|
||||
- **Not a schema builder.** The typedef engine does not provide a fluent
|
||||
API for constructing schemas. Schemas are plain JSON — authored in
|
||||
TypeBox, generated by ujsx components, or hand-written. A builder API
|
||||
is deferred (OQ-071).
|
||||
is deferred (OQ-003).
|
||||
- **Not a serialization framework.** The typedef engine is not a
|
||||
general-purpose serde replacement. It operates on raw byte buffers at
|
||||
computed offsets — no intermediate `Value` tree, no reflection, no
|
||||
@@ -171,22 +171,22 @@ These boundaries are decided in [ADR-095](decisions/095-alknet-typedef-purpose-s
|
||||
|
||||
| Decision | ADR | Summary |
|
||||
|----------|-----|---------|
|
||||
| Purpose, scope, and the jsonschema engine | [ADR-095](decisions/095-alknet-typedef-purpose-scope-jsonschema-engine.md) | What the crate is/isn't; why jsonschema not a custom engine; "schema is the format" principle; scope boundaries |
|
||||
| Two layout modes | [ADR-096](decisions/096-two-layout-modes-packed-vs-aligned.md) | Packed sequential (`LayoutBuilder`/`SequentialReader`) for protocols; aligned static (`OffsetMap`) for mmap formats |
|
||||
| Schema annotations | [ADR-097](decisions/097-schema-annotations.md) | Endianness (schema-level, default LE), alignment (struct + field-level), encoding (length-prefixed vs offset-indirect), TUnion discriminators (byte-offset vs field-name) |
|
||||
| Error handling and validation | [ADR-098](decisions/098-error-handling-validation-strategy.md) | `TypedefError` enum; load-time build, access-time check; field-path-carrying errors; jsonschema `ValidationError` wrapping |
|
||||
| Int64/Uint64 kinds | [ADR-099](decisions/099-int64-uint64-first-class-kinds.md) | 64-bit integers as first-class kinds (SFTP offsets, metatensor data_offsets) |
|
||||
| Non-final inline variable fields | [ADR-100](decisions/100-reject-non-final-inline-length-prefixed-in-aligned-mode.md) | Rejected in aligned mode (would clobber subsequent fields) |
|
||||
| Packed-mode read factory | [ADR-101](decisions/101-packed-mode-read-factory.md) | `engine.sequential_reader()` returns an owned fresh reader |
|
||||
| TUnion in aligned mode | [ADR-102](decisions/102-reject-tunion-in-aligned-mode.md) | Rejected for v1 (broken semantics; no current consumer needs it) |
|
||||
| Purpose, scope, and the jsonschema engine | [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md) | What the crate is/isn't; why jsonschema not a custom engine; "schema is the format" principle; scope boundaries |
|
||||
| Two layout modes | [ADR-002](decisions/002-two-layout-modes-packed-vs-aligned.md) | Packed sequential (`LayoutBuilder`/`SequentialReader`) for protocols; aligned static (`OffsetMap`) for mmap formats |
|
||||
| Schema annotations | [ADR-003](decisions/003-schema-annotations.md) | Endianness (schema-level, default LE), alignment (struct + field-level), encoding (length-prefixed vs offset-indirect), TUnion discriminators (byte-offset vs field-name) |
|
||||
| Error handling and validation | [ADR-004](decisions/004-error-handling-validation-strategy.md) | `TypedefError` enum; load-time build, access-time check; field-path-carrying errors; jsonschema `ValidationError` wrapping |
|
||||
| Int64/Uint64 kinds | [ADR-005](decisions/005-int64-uint64-first-class-kinds.md) | 64-bit integers as first-class kinds (SFTP offsets, metatensor data_offsets) |
|
||||
| Non-final inline variable fields | [ADR-006](decisions/006-reject-non-final-inline-length-prefixed-in-aligned-mode.md) | Rejected in aligned mode (would clobber subsequent fields) |
|
||||
| Packed-mode read factory | [ADR-007](decisions/007-packed-mode-read-factory.md) | `engine.sequential_reader()` returns an owned fresh reader |
|
||||
| TUnion in aligned mode | [ADR-008](decisions/008-reject-tunion-in-aligned-mode.md) | Rejected for v1 (broken semantics; no current consumer needs it) |
|
||||
|
||||
## Open Questions
|
||||
|
||||
See [open-questions.md](open-questions.md) for full details.
|
||||
|
||||
- **OQ-069** (deferred(scope)): Arrays of variable-length-element structs.
|
||||
- **OQ-070** (deferred(scope)): `no_std` + `alloc` support.
|
||||
- **OQ-071** (deferred(scope)): Builder API for schema construction.
|
||||
- **OQ-001** (deferred(scope)): Arrays of variable-length-element structs.
|
||||
- **OQ-002** (deferred(scope)): `no_std` + `alloc` support.
|
||||
- **OQ-003** (deferred(scope)): Builder API for schema construction.
|
||||
|
||||
## References
|
||||
|
||||
@@ -200,4 +200,4 @@ See [open-questions.md](open-questions.md) for full details.
|
||||
- `/workspace/jsonschema/` — the jsonschema crate (v0.46.5, Draft 2020-12)
|
||||
- `/workspace/alknet-typedef-poc/` — the POC code (disposable)
|
||||
- `/workspace/@alkimiadev/typebox-rs/` — prior attempt, replaced by typedef
|
||||
- `/workspace/@alkimiadev/alktype/` — prior attempt, replaced by typedef
|
||||
- `/workspace/@alkimiadev/alktype/` — prior attempt (the @alkimiadev/alktype prototype; not to be confused with this crate, which reuses the name but is backed by the `jsonschema` crate)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# OQ-069: Arrays of variable-length-element structs
|
||||
# OQ-001: Arrays of variable-length-element structs
|
||||
|
||||
- **Origin**: [../layout-engine.md](../layout-engine.md),
|
||||
[../data-access.md](../data-access.md);
|
||||
@@ -17,4 +17,4 @@
|
||||
walking of array elements, reading each element's length prefixes to
|
||||
find the next element's start) is understood but not needed by any
|
||||
current consumer. Arrays of fixed-size structs are fully supported.
|
||||
- **Cross-references**: ADR-096, [layout-engine.md](../layout-engine.md)
|
||||
- **Cross-references**: ADR-002, [layout-engine.md](../layout-engine.md)
|
||||
@@ -1,4 +1,4 @@
|
||||
# OQ-070: `no_std` + `alloc` support
|
||||
# OQ-002: `no_std` + `alloc` support
|
||||
|
||||
- **Origin**: [../overview.md](../overview.md);
|
||||
`docs/research/alknet-typedef/findings.md` §"Open Questions" (OQ 6)
|
||||
@@ -19,4 +19,4 @@
|
||||
later. The engine's core (offset computation, read/write) is already
|
||||
allocation-free — it operates on `&[u8]` slices. The `jsonschema`
|
||||
dependency is the only `alloc` consumer.
|
||||
- **Cross-references**: ADR-095
|
||||
- **Cross-references**: ADR-001
|
||||
@@ -1,4 +1,4 @@
|
||||
# OQ-071: Builder API for schema construction
|
||||
# OQ-003: Builder API for schema construction
|
||||
|
||||
- **Origin**: [../schema-layer.md](../schema-layer.md),
|
||||
[../overview.md](../overview.md);
|
||||
@@ -23,4 +23,4 @@
|
||||
path is the primary interface for v1. A builder API would be a fluent
|
||||
Rust API that produces the same JSON Schema structure — it would sit
|
||||
on top of the engine, not inside it.
|
||||
- **Cross-references**: ADR-095, [schema-layer.md](../schema-layer.md)
|
||||
- **Cross-references**: ADR-001, [schema-layer.md](../schema-layer.md)
|
||||
@@ -3,7 +3,7 @@ status: draft
|
||||
last_updated: 2026-07-22
|
||||
---
|
||||
|
||||
# alknet-typedef — Schema Layer
|
||||
# alktype — Schema Layer
|
||||
|
||||
The schema layer: the 19 `TypeDef:*` custom type kinds, their mapping to
|
||||
Rust types and byte sizes, the `jsonschema` custom keyword integration,
|
||||
@@ -40,11 +40,11 @@ encoding strategy (for variable-length types).
|
||||
| `TRecord` | `TypeDef:Record` | count-prefixed sequence of (key, value) pairs | variable | variable |
|
||||
| `TTimestamp` | `TypeDef:Timestamp` | length-prefixed RFC 3339 string | variable | variable |
|
||||
|
||||
`TypeDef:Int64` and `TypeDef:Uint64` are alknet-typedef additions —
|
||||
`TypeDef:Int64` and `TypeDef:Uint64` are alktype additions —
|
||||
TypeBox's `typedef.ts` tops out at 32-bit integers. They are required by
|
||||
the primary POC targets: SFTP `Read`/`Write` packets have `offset: u64`,
|
||||
and metatensor `data_offsets` are `u64`. See
|
||||
[ADR-099](decisions/099-int64-uint64-first-class-kinds.md).
|
||||
[ADR-005](decisions/005-int64-uint64-first-class-kinds.md).
|
||||
|
||||
### The `TypeDefKind` enum
|
||||
|
||||
@@ -106,7 +106,7 @@ number of variants (e.g., the call protocol's 5 event types); a `u32`
|
||||
index is compact, fixed-size, and sufficient for any realistic enum. The
|
||||
JSON representation (for validation) remains a string; the binary
|
||||
representation is the `u32` index.
|
||||
The `u32` index follows the schema's endianness annotation (ADR-097), like
|
||||
The `u32` index follows the schema's endianness annotation (ADR-003), like
|
||||
all other fixed-size types. In little-endian mode the index is
|
||||
`u32::from_le_bytes`; in big-endian mode it is `u32::from_be_bytes`.
|
||||
|
||||
@@ -161,7 +161,7 @@ length prefixes and without reserving worst-case space.
|
||||
length-prefixing) otherwise.
|
||||
|
||||
**Length prefix endianness:** The 4-byte length prefix (strategies 1 and 3)
|
||||
respects the schema's `"endian"` annotation (ADR-097). In little-endian
|
||||
respects the schema's `"endian"` annotation (ADR-003). In little-endian
|
||||
mode, the length is `u32::from_le_bytes`. In big-endian mode, the length
|
||||
is `u32::from_be_bytes`. This ensures SFTP consumers (big-endian) have
|
||||
consistent byte order for both field values and length prefixes.
|
||||
@@ -169,7 +169,7 @@ consistent byte order for both field values and length prefixes.
|
||||
**`TBytes`:** Raw bytes — no UTF-8 constraint. The payload is `&[u8]`.
|
||||
Otherwise identical to `TString` in layout (same three strategies).
|
||||
|
||||
**Design note:** `TypeDef:Bytes` is an alknet-typedef addition — it does
|
||||
**Design note:** `TypeDef:Bytes` is an alktype addition — it does
|
||||
not exist in TypeBox's `typedef.ts` (which defines 16 kinds). It is
|
||||
included because raw byte arrays are a common binary protocol primitive
|
||||
(SFTP data payloads, channels payloads, tensor data) and are semantically
|
||||
@@ -360,7 +360,7 @@ a ujsx component, or hand-written. The schema is the interface.
|
||||
## Schema Annotations
|
||||
|
||||
Schema-level annotations control binary layout behavior. These are
|
||||
decided in [ADR-097](decisions/097-schema-annotations.md).
|
||||
decided in [ADR-003](decisions/003-schema-annotations.md).
|
||||
|
||||
### Endianness
|
||||
|
||||
@@ -393,7 +393,7 @@ Both struct-level and field-level, with field-level overriding:
|
||||
- Default alignment: 1 for u8/i8/bool, 2 for u16/i16, 4 for u32/i32/f32/
|
||||
enum, 8 for u64/i64/f64, 4 for variable-length (the u32 length prefix),
|
||||
1 for struct/union/array.
|
||||
- Only meaningful in aligned static mode (ADR-096). Ignored in packed
|
||||
- Only meaningful in aligned static mode (ADR-002). Ignored in packed
|
||||
sequential mode.
|
||||
|
||||
### Variable-length encoding
|
||||
@@ -486,21 +486,21 @@ Mapping values may be either inline schemas or `$ref` pointers. Both work.
|
||||
|
||||
| Decision | ADR | Summary |
|
||||
|----------|-----|---------|
|
||||
| Schema annotations | [ADR-097](decisions/097-schema-annotations.md) | Concrete JSON shapes for endianness, alignment, encoding, and TUnion discriminators |
|
||||
| Int64/Uint64 kinds | [ADR-099](decisions/099-int64-uint64-first-class-kinds.md) | 64-bit integers as first-class kinds (required by SFTP offsets and metatensor data_offsets) |
|
||||
| Purpose and scope | [ADR-095](decisions/095-alknet-typedef-purpose-scope-jsonschema-engine.md) | Why jsonschema not a custom engine; "schema is the format" principle |
|
||||
| Schema annotations | [ADR-003](decisions/003-schema-annotations.md) | Concrete JSON shapes for endianness, alignment, encoding, and TUnion discriminators |
|
||||
| Int64/Uint64 kinds | [ADR-005](decisions/005-int64-uint64-first-class-kinds.md) | 64-bit integers as first-class kinds (required by SFTP offsets and metatensor data_offsets) |
|
||||
| Purpose and scope | [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md) | Why jsonschema not a custom engine; "schema is the format" principle |
|
||||
|
||||
## Open Questions
|
||||
|
||||
See [open-questions.md](open-questions.md) for full details.
|
||||
|
||||
- **OQ-071** (deferred(scope)): Builder API for schema construction.
|
||||
- **OQ-003** (deferred(scope)): Builder API for schema construction.
|
||||
|
||||
## References
|
||||
|
||||
- `/workspace/@alkdev/typebox/example/typedef/typedef.ts` — the TypeBox
|
||||
schema kinds (619 lines)
|
||||
- `/workspace/jsonschema/` — the jsonschema crate (v0.46.5, Draft 2020-12)
|
||||
- [ADR-097](decisions/097-schema-annotations.md) — schema
|
||||
- [ADR-003](decisions/003-schema-annotations.md) — schema
|
||||
annotation shapes
|
||||
- [validation.md](validation.md) — custom keyword validator implementations
|
||||
|
||||
@@ -3,7 +3,7 @@ status: draft
|
||||
last_updated: 2026-07-22
|
||||
---
|
||||
|
||||
# alknet-typedef — Validation
|
||||
# alktype — Validation
|
||||
|
||||
The validation layer: custom keyword validators for all 19 `TypeDef:*`
|
||||
kinds, the `TypedefError` enum, load-time vs access-time validation
|
||||
@@ -17,7 +17,7 @@ it registers custom keyword validators for each `TypeDef:*` kind and
|
||||
lets `jsonschema` handle the structural validation (object properties,
|
||||
required fields, array items, enum values).
|
||||
|
||||
The strategy is decided in [ADR-098](decisions/098-error-handling-validation-strategy.md):
|
||||
The strategy is decided in [ADR-004](decisions/004-error-handling-validation-strategy.md):
|
||||
|
||||
1. **Load time:** Parse the schema JSON, build the layout engine, build the
|
||||
jsonschema validator. This is the `TypedefEngine::compile(schema)` constructor.
|
||||
@@ -50,7 +50,7 @@ process, not a single `validate(buffer)` call.
|
||||
### The `TypedefEngine` struct
|
||||
|
||||
The `TypedefEngine` is the compiled form of a schema. It supports both
|
||||
layout modes (ADR-096) via an internal `Layout` enum:
|
||||
layout modes (ADR-002) via an internal `Layout` enum:
|
||||
|
||||
```rust
|
||||
pub struct TypedefEngine {
|
||||
@@ -78,7 +78,7 @@ impl TypedefEngine {
|
||||
pub fn endian(&self) -> Endian;
|
||||
pub fn offset_map(&self) -> Option<&OffsetMap>; // Some in aligned mode
|
||||
pub fn layout_builder(&self) -> Option<&LayoutBuilder>; // Some in packed mode
|
||||
pub fn sequential_reader(&self) -> Option<SequentialReader>; // owned fresh reader (ADR-101)
|
||||
pub fn sequential_reader(&self) -> Option<SequentialReader>; // owned fresh reader (ADR-007)
|
||||
}
|
||||
```
|
||||
|
||||
@@ -92,7 +92,7 @@ mode-agnostic (it operates on `Value`, not raw bytes).
|
||||
The `Layout::Packed` variant stores only the `LayoutBuilder` (write-side).
|
||||
The `SequentialReader` (read-side) is not stored — it has mutable cursor
|
||||
state that the consumer owns, so `sequential_reader()` constructs a fresh
|
||||
reader on each call (ADR-101).
|
||||
reader on each call (ADR-007).
|
||||
|
||||
The `read_field`/`write_field` methods on `TypedefEngine` are the
|
||||
aligned-mode data-access API — see [data-access.md](data-access.md)
|
||||
@@ -219,7 +219,7 @@ example, a `TypeDef:Struct` validator can inspect the parent's
|
||||
|
||||
A single `TypedefError` enum covers all error conditions across the
|
||||
engine's three phases (schema parsing, offset computation, read/write)
|
||||
plus validation. Decided in [ADR-098](decisions/098-error-handling-validation-strategy.md).
|
||||
plus validation. Decided in [ADR-004](decisions/004-error-handling-validation-strategy.md).
|
||||
|
||||
```rust
|
||||
pub enum TypedefError {
|
||||
@@ -314,20 +314,20 @@ representation first, then access the binary buffer.
|
||||
|
||||
| Decision | ADR | Summary |
|
||||
|----------|-----|---------|
|
||||
| Error handling and validation | [ADR-098](decisions/098-error-handling-validation-strategy.md) | `TypedefError` enum; load-time build, access-time check; field-path-carrying errors; jsonschema `ValidationError` wrapping |
|
||||
| Purpose and scope | [ADR-095](decisions/095-alknet-typedef-purpose-scope-jsonschema-engine.md) | Why jsonschema not a custom engine |
|
||||
| Error handling and validation | [ADR-004](decisions/004-error-handling-validation-strategy.md) | `TypedefError` enum; load-time build, access-time check; field-path-carrying errors; jsonschema `ValidationError` wrapping |
|
||||
| Purpose and scope | [ADR-001](decisions/001-alktype-purpose-scope-jsonschema-engine.md) | Why jsonschema not a custom engine |
|
||||
|
||||
## Open Questions
|
||||
|
||||
None specific to validation. The three typedef OQs (OQ-069, OQ-070,
|
||||
OQ-071) are about layout, platform support, and schema construction —
|
||||
None specific to validation. The three typedef OQs (OQ-001, OQ-002,
|
||||
OQ-003) are about layout, platform support, and schema construction —
|
||||
not validation.
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/alknet-typedef/findings.md` §"Validation" — the POC's
|
||||
custom keyword validators for all 17 kinds
|
||||
- [ADR-098](decisions/098-error-handling-validation-strategy.md) —
|
||||
- [ADR-004](decisions/004-error-handling-validation-strategy.md) —
|
||||
error handling and validation strategy
|
||||
- [schema-layer.md](schema-layer.md) — the 17 TypeDef kinds that the
|
||||
validators check
|
||||
|
||||
Reference in New Issue
Block a user