alktype/CHANGELOG.md

# Changelog

All notable changes to this crate are documented here. The format is
based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and
this crate adheres to [Semantic Versioning](https://semver.org/).

## [0.3.0] - 2026-09-07

The compiled-forms release. The packed read path — the hot path for
stream parsing — is driven by a compile-once `ReadPlan` instead of a
per-call walk of the BAST typed tree; byte validation runs a compiled
`ValidationPlan`; plans and offset maps fingerprint to a stable hash
(ADR-011/ADR-012). Reads of SFTP-shaped packet streams went from
~189× hand-rolled Rust to ~74× (~2.4× faster), fixed-stride chunk
reads from ~18× to ~11×, and `SequentialReader::read_next_borrowed`
makes the per-field hot loop allocation-free.

### Breaking changes

- **`Bast*` types are owned.** `BastDoc`/`BastStruct`/`BastField`/… no
  longer borrow from the source `serde_json::Value`; all v0.2.0
  lifetimes are gone. `BastDoc::new` parses the root eagerly; `$ref`s
  resolve lazily.
- **`OffsetMap::get` / `PackedLayout::get`** return `&OffsetEntry`
  (was `Option<OffsetEntry>` by value), backed by an O(log n)
  `BTreeMap` path→index (first-occurrence-wins for duplicate names).
- **`SequentialReader::new`** takes the compiled plan; construct via
  `AlkTypeEngine::sequential_reader()` (packed mode only).
- **`materialize_packed` / `materialize_aligned`** take the compiled
  plan / `(&BastDoc, &OffsetMap)` pair respectively.
- **Field-name-discriminator union wire convention** (ADR-011
  addendum): the builder lays out the union's declared `fields`
  (shared) first, then the variant's own fields. Variants must not
  re-declare the discriminator or any shared field, and the
  discriminator field must be the first entry in `fields` — all
  enforced at parse with clean `Schema` errors. Schemas relying on
  0.2.0's variant-only layout are rejected (they produced
  reader↔builder-disagreeing bytes).
- **`maxLength` is string/bytes-only** — rejected at parse on every
  other kind (it was silently unenforced there).
- **Aligned-mode `Record` fields reject `offset-indirect`** (the
  materializer always walks the inline count-prefixed form — the
  annotated shape was never readable).
- **Schema input bounds** (untrusted-schema hardening, AGENTS.md §3):
  array `count` ≤ 2^16 and `count × stride` ≤ 2^26 bytes; `align` ≤
  4096; `maxLength` ≤ 2^26; cyclic `$ref` graphs and >128-deep nesting
  are rejected by every public walker (`OffsetMap::compute`,
  `LayoutBuilder::new`, `materialize_aligned` included), not just the
  engine.

### Additions

- **`ReadPlan`** (ADR-011) — the compiled packed-read plan, re-exported
  with `CompositePlan`/`FieldPlan`/`ReadKind`/`DiscriminatorPlan`.
  `ReadPlan::compile` is untrusted-input-safe standalone (depth cap +
  cycle set). `fixed_size()` exposes the compile-time-known byte size
  for fixed structs.
- **`ValidationPlan`** (ADR-012 §3) — the compiled `validate_bytes`
  walker, with `ValidNode`/`ValidVariant` sub-types.
- **`fingerprint()`** on `ReadPlan`/`OffsetMap`/`ValidationPlan` +
  `Hash`/`Eq` derives on the plan types (ADR-012 §1/§4) — plan
  identity for cache-keying across processes.
- **`OffsetMap` `LeafMeta`** — each entry records whether it is
  fixed/length-prefixed/offset-indirect so `read_field`/`write_field`
  dispatch without re-walking the schema; `OffsetEntry` type re-exported.
- **`SequentialReader::read_next_borrowed`** — zero-allocation variant
  of `read_next` (field name borrowed from the plan).
- **`AlkTypeEngine::validate_bytes`** now runs the compiled
  `ValidationPlan` (was an interpretive BAST walk in 0.2.0).

### Fixes (post-release-commit hardening — reviews #006, #007, #008)

All found and fixed before the first crates.io publish of 0.3.0, so
no published version ever exhibited them.

- **Untrusted-input crashes removed.** A huge declared array count
  OOM-aborted the process (`Vec::with_capacity(count)` before reading
  a byte) — now compile-capped and walked with push-only growth.
  Cyclic `$ref` graphs stack-overflowed the three standalone layout
  walkers — now guarded by a shared reference-graph check. Deeply
  nested stride-0 arrays briefly allowed ~477 MB of simultaneous
  allocation from a ~1 KB schema — restored to incremental growth.
- **Cross-consumer divergences closed.** Builder, reader,
  materializer, tunion, and the validation plan now agree on
  field-disc union layout (shared-then-variant), on the discriminator
  field's position (must be first), and on union mapping-key matching
  (numeric fast-path dispatch only for canonical keys like `"2"`;
  `"01"`/`"+1"` fall back to the string comparison all consumers
  share). The legacy BAST walker's field-disc union arm walks shared
  fields before the variant (it previously materialized variant fields
  from shared fields' bytes).
- **Silently-corrupt layouts rejected.** Aligned record fields with
  `maxLength`/`offset-indirect`; non-final inline length-prefixed
  fields (records included — the ADR-006 check now sees them);
  aligned-mode `maxLength`/`offset-indirect` on records; unions in
  aligned mode (pre-existing, now tested).
- **Coverage**: 90.67% lines / 86.32% functions at review #007's
  audit, 91.66% after its fixes; every uncovered region outside test
  modules read and classified in-tree (docs/reviews/007).

### Non-breaking improvements

- Engine compile is one-shot and allocation-tidy; plans are
  `Send + Sync` (statically asserted) and fingerprintable.
- Zero-progress array-element guard on all three array walkers (a
  zero-size element makes the declared count unbounded on the wire).
- WASM-clean unchanged: two dependencies (`jsonschema`
  default-features off, `serde_json` with `preserve_order`), no
  `async`, no `unsafe`, no feature flags.
- Benches (`benches/wire_vs_bast.rs`): read/write chunk streams, an
  SFTP-shaped union packet stream, and `validate_bytes` per buffer —
  the numbers quoted above and in ADR-007/ADR-011.

## [0.2.0] - 2026-08-17

A breaking release that replaces the v0.1.0 `AlkType:*` custom-keyword
JSON Schema format with BAST (Binary Abstract Syntax Tree) — a JSON
document that describes binary layouts using a `kind`-based vocabulary
with `$defs`/`$ref` for composition. BAST is itself a valid JSON Schema
instance (it has a meta-schema), making it self-validating,
editor-friendly, and trivially consumable from any language with a JSON
parser. The engine works the same way as before: compile a document
once into an `AlkTypeEngine`, then read/write fields at computed offsets
and validate bytes/JSON. The pivot was made now because v0.1.0 has no
real consumers (≈15 crates.io downloads, mostly bots/scanners), so the
custom-keyword wart could be removed cleanly.

### Breaking changes

- **Schema format.** The v0.1.0 `AlkType:*` custom-keyword JSON Schema
  format (`{ "AlkType:Struct": true, "fields": [...] }`) is removed.
  Schemas are now BAST documents:
  `{ "$defs": { "<TypeName>": { "kind": "struct", "fields": [...] } } }`.
  The `kind`-based vocabulary covers 18 binary kinds (integers, floats,
  bytes, string, struct, union, enum, array, etc.).
- **`AlkTypeEngine::compile` signature.** Now takes
  `(bast_doc: &Value, root_name: &str, mode: LayoutMode, json_schema: Option<&Value>)`.
  The root type name is a required parameter — it selects which `$defs`
  entry is the top-level type (previously the root was implicit from the
  single top-level schema object).
- **Builder API output.** `Definitions`/`Schema`/`Discriminator` now
  produce BAST JSON via `Definitions::build_doc(name, schema)`. The
  builder method names are unchanged; only the emitted JSON shape
  changed. `Schema::struct_()` produces a BAST struct;
  `Schema::object()` produces a standard JSON Schema (for the
  `validate_json` path).
- **Validation split.** v0.1.0 used a single `jsonschema` validator
  with 19 custom `AlkType:*` keywords for both bytes and JSON
  validation. 0.2.0 splits this into two independent paths:
  - `validate_bytes` uses a new BAST-native validator
    (`bast_validation`) — a recursive walker over the BAST type tree.
  - `validate_json` / `is_valid_json` use a standard
    `jsonschema::Validator` built from a consumer-provided JSON Schema
    (passed to `compile` as the `json_schema` parameter). No custom
    keywords; BAST is not involved — BAST describes bytes, not JSON
    shape.
  Both paths return `AlkTypeError::Validation` with a uniform
  `jsonschema::ValidationError<'static>` payload.
- **Removed.** The v0.1.0 custom-keyword accessor layer
  (`AlkTypeKind::FromStr`, `parse_*`, `resolve_ref*`,
  `DiscriminatorKind`) is removed. The BAST parser (`bast` module)
  exposes a cleaner typed surface (`BastDoc`/`BastDef`/`BastStruct`/
  `BastField`/`BastType`/etc.) that borrows from the source
  `serde_json::Value` without cloning field data.
- **Public module surface.** New public modules: `bast`, `bast_meta`,
  `bast_validation`, `builder`, `materialize`. The `schema` module is
  retained but now holds only `Endian`/`AlkTypeKind`/`VariableEncoding`
  (the binary-kind vocabulary); the v0.1.0 custom-keyword machinery is
  gone.

### Additions

- **BAST meta-schema.** Embedded in the crate as
  `BAST_META_SCHEMA` (re-exported from the crate root) and published at
  `https://alk.dev/bast/v1/schema`. BAST documents are validated against
  it at compile time (`AlkTypeEngine::compile` calls
  `validate_bast_doc` before parsing).
- **`materialize` module.** Materializes a `serde_json::Value` tree from
  a binary buffer by walking the BAST typed tree. Used by
  `AlkTypeEngine::validate_bytes` (ADR-010).
- **Builder for JSON Schemas.** `Schema::object()` produces a standard
  JSON Schema object (for the `validate_json` path), complementing
  `Schema::struct_()` which produces a BAST struct (for the bytes path).
  One builder, two output shapes — the method name selects which.

### Bug fixes vs v0.1.0

- **Enum index bounds are now checked.** The v0.1.0 validator had a
  dead constraint: enum variant indices were never bounds-checked
  against `values.len()`. The BAST-native validator enforces it
  (`validate_enum` checks `idx < values.len()`).
- **Offset-indirect, field-level endian, and aligned materialization**
  bugs found during review #003 are fixed.

### Non-breaking improvements

- `$ref` is restricted to `#/$defs/<name>` — one hash lookup, no
  `normalize_refs` pass (the v0.1.0 engine needed one).
- Schemas remain untrusted input: every engine path that walks a BAST
  document returns `Err` on a malformed document, never `panic!`/
  `unreachable!`. Overflow-safe arithmetic (`checked_add`,
  `usize::try_from`) on all offset/count casts.
- Still two dependencies (`jsonschema` with `default-features = false`,
  `serde_json` with `preserve_order`), no `async`, no `unsafe`, no
  platform deps, no feature flags. Compiles to
  `wasm32-unknown-unknown`.

### Upgrade notes

There is no migration path from v0.1.0 `AlkType:*` schemas — the format
is incompatible. Rewrite schemas as BAST documents (the `builder` API
produces them; see the README usage example) and update `compile` calls
to pass the root type name and the optional JSON Schema. The
read/write/validate API surface (`read_field`, `write_field`,
`sequential_reader`, `validate_bytes`, `validate_json`,
`is_valid_json`) is unchanged.

## [0.1.0] - 2025-11-10

Initial crates.io release. Custom-keyword JSON Schema format
(`AlkType:*`), single `jsonschema` validator for both bytes and JSON,
`AlkTypeEngine` with packed/aligned layout modes, builder API producing
`serde_json::Value`.

[0.3.0]: https://git.alk.dev/alkdev/alktype/releases/tag/v0.3.0
[0.2.0]: https://git.alk.dev/alkdev/alktype/releases/tag/v0.2.0
[0.1.0]: https://git.alk.dev/alkdev/alktype/releases/tag/v0.1.0