From 9949f914df4bcfecb52e0f92159d41ec011ee976 Mon Sep 17 00:00:00 2001 From: "glm-5.3-flash" Date: Wed, 2 Sep 2026 09:24:30 +0000 Subject: [PATCH] =?UTF-8?q?Release=20v0.3.0:=20compiled=20forms=20?= =?UTF-8?q?=E2=80=94=20ReadPlan,=20owned=20BastDoc,=20LeafMeta,=20Validati?= =?UTF-8?q?onPlan,=20fingerprinting?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Public API bump 0.2.0 -> 0.3.0 (the 030-compiled-forms plan is now fully implemented; all eight phases landed). - Cargo.toml: version 0.3.0. lib.rs re-exports complete (ReadPlan + sub-types, LeafMeta, OffsetEntry, ValidationPlan + sub-types). - ADR-007 "Cost" rewritten to the Arc cost (15.7 ns) with the 0.2.0 "re-parse on demand" framing as a historical note (review #004 L2, the last loose end from that review). - ADR-011/012 status blocks flipped to implemented; architecture README ADR table rows updated; layout-engine.md rewritten for the 0.3.0 surface (engine-factory reader construction, OffsetMap OffsetEntry/LeafMeta/fingerprint section, owned BastDoc compute signature); SequentialReader module doc points at the engine factory. Reviews #004 and #005 flipped to closed. - Bench re-run (alktty wire_vs_bast, 0.3.0 tree): read p64 98 ns/chunk (parity with phase 2; hand-rolled 5.7 us/stream), layout_build 180 ns (was ~1.2 us — the phase-4 owned-doc cache removed the per-build re-parse, ~7x), sequential_reader_new 15.7 ns, write p64 -3%, engine_compile unchanged (meta-schema validation dominates). No dedicated validate_bytes-stream bench: the phase-7 spot check (~0.2 us plan-validate vs ~0.6 us compile-per-call) stands; a dedicated bench is a follow-up if alkcall profiling motivates it. - Downstream: alktty compiles against the path dep unchanged; alkcall has no dependency yet. Verification (full block, all green): 474 tests; clippy -D warnings clean; cargo doc zero warnings; wasm32 release build green; cargo publish --dry-run clean at 0.3.0. --- Cargo.lock | 2 +- Cargo.toml | 2 +- docs/architecture/README.md | 4 +- .../decisions/007-packed-mode-read-factory.md | 33 ++++++++-------- .../011-compiled-read-plan-for-packed-mode.md | 3 +- .../012-plan-fingerprinting-and-m1-closure.md | 6 ++- docs/architecture/layout-engine.md | 38 +++++++++++++++---- docs/plans/030-compiled-forms.md | 31 ++++++++++++++- docs/reviews/004-performance-review.md | 4 +- docs/reviews/005-plan-review-030.md | 4 +- src/sequential_reader.rs | 3 +- 11 files changed, 94 insertions(+), 36 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 85d78be..b3d547b 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -27,7 +27,7 @@ dependencies = [ [[package]] name = "alktype" -version = "0.2.0" +version = "0.3.0" dependencies = [ "jsonschema", "serde_json", diff --git a/Cargo.toml b/Cargo.toml index d794a25..67a16b6 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "alktype" -version = "0.2.0" +version = "0.3.0" edition = "2021" rust-version = "1.85" license = "MIT OR Apache-2.0" diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 45d739b..1909797 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -45,8 +45,8 @@ format definition; the engine is generic. | [008](decisions/008-reject-tunion-in-aligned-mode.md) | Reject TUnion in Aligned Mode for v1 | Unions are the protocol pattern; aligned-mode union semantics were broken | | [009](decisions/009-builder-api.md) | Builder API for Schema Construction | Fluent Rust API producing `serde_json::Value`; covers BAST kinds + standard JSON Schema; resolves OQ-003. *Output format amended to BAST / standard JSON Schema by ADR-BAST.* | | [010](decisions/010-generalized-validation-validate-bytes.md) | Generalized Validation — `validate_bytes` on `AlkTypeEngine` | Single-call binary-buffer validation; materialize `Value` from bytes, then validate. *Validation step amended to the BAST-native validator by ADR-VAL-SPLIT.* | -| [011](decisions/011-compiled-read-plan-for-packed-mode.md) | Compiled Read Plan for Packed Mode | `ReadPlan` — the packed read-side compiled form, symmetric to `OffsetMap` (aligned) and `PackedLayout` (packed write). Closes review #004's 400x read-path gap; retires ADR-007's "re-parse on demand" framing. *Accepted.* | -| [012](decisions/012-plan-fingerprinting-and-m1-closure.md) | Plan Fingerprinting, ValidationPlan, and Closing the Deferred M1 Sites in 0.3.0 | `ReadPlan`/`OffsetMap`/`ValidationPlan` `Hash + Eq` + `fingerprint()`; owned `BastDoc` (lifetime removal); `OffsetMap` carries `LeafMeta` to close the aligned-side M1 sites; `ValidationPlan` retires the interpretive `bast_validation` walk (review #005 M3 reversed the original deferral). Bundles with ADR-011 into one 0.3.0 breaking release. *Accepted — §3a shape implemented in `src/validation_plan.rs`.* | +| [011](decisions/011-compiled-read-plan-for-packed-mode.md) | Compiled Read Plan for Packed Mode | `ReadPlan` — the packed read-side compiled form, symmetric to `OffsetMap` (aligned) and `PackedLayout` (packed write). Closes review #004's 400x read-path gap; retires ADR-007's "re-parse on demand" framing. *Accepted — implemented in 0.3.0 (phases 1–2).* | +| [012](decisions/012-plan-fingerprinting-and-m1-closure.md) | Plan Fingerprinting, ValidationPlan, and Closing the Deferred M1 Sites in 0.3.0 | `ReadPlan`/`OffsetMap`/`ValidationPlan` `Hash + Eq` + `fingerprint()`; owned `BastDoc` (lifetime removal); `OffsetMap` carries `LeafMeta` to close the aligned-side M1 sites; `ValidationPlan` retires the interpretive `bast_validation` walk (review #005 M3 reversed the original deferral). Bundles with ADR-011 into one 0.3.0 breaking release. *Accepted — fully implemented in 0.3.0 (fingerprinting, owned `BastDoc`, `LeafMeta`, `ValidationPlan`).* | ## Relevant Open Questions diff --git a/docs/architecture/decisions/007-packed-mode-read-factory.md b/docs/architecture/decisions/007-packed-mode-read-factory.md index ee2bada..3de7df1 100644 --- a/docs/architecture/decisions/007-packed-mode-read-factory.md +++ b/docs/architecture/decisions/007-packed-mode-read-factory.md @@ -63,23 +63,26 @@ write-side. ### Cost -`SequentialReader::new` clones the top-level struct's field schemas (a -`Vec<(String, Value)>` of the `properties` entries) and clones the -schema itself. This is cheap — a struct has a small number of fields -(SFTP's largest packet has 5). The construction cost is negligible -compared to the cost of reading a buffer. +`SequentialReader::new(Arc)` is a refcount bump — 15.7 ns +(measured, alktty `wire_vs_bast` bench, 0.3.0). The reader shares the +engine's compiled [`ReadPlan`](011-compiled-read-plan-for-packed-mode.md) +(the packed read-side compiled form) via `Arc` instead of cloning +schema data; construction cost is negligible compared to reading a +buffer. The engine holds the owned `BastDoc` (ADR-012 §2a) for the +aligned materialize path and the one-shot `*::compile` paths. -> **Note**: The "re-parse on demand" framing below (the read loop -> re-parsing `BastDoc::new` per field) is the root cause of the 400x -> read-path gap measured in +> **Historical note**: the original 0.2.0 framing here ("re-parse on +> demand" — the read loop re-parsing `BastDoc::new` per field) was the +> root cause of the 400x read-path gap measured in > [review #004](../../reviews/004-performance-review.md). -> [ADR-011](011-compiled-read-plan-for-packed-mode.md) (Proposed) -> retires this framing by giving the packed read path a compiled -> `ReadPlan`; the "Cost" section here and the -> `src/engine.rs:112-115` doc comment will be updated in the -> implementation commit per ADR-011's recommended order. The factory -> decision itself (`sequential_reader() -> Option`, -> owned fresh reader, consumer-driven cursor) is retained. +> [ADR-011](011-compiled-read-plan-for-packed-mode.md) (implemented, +> 0.3.0) retired it: the packed read loop walks `Arc` (2.27 +> µs/chunk → 98 ns/chunk), `sequential_reader()` is an `Arc::clone`, +> and the owned `BastDoc` (ADR-012 §2a) removed the remaining +> per-access re-parse sites in `read_field`/`write_field`/`validate_bytes`. +> The factory decision itself (`sequential_reader() -> +> Option`, owned fresh reader, consumer-driven +> cursor) was retained unchanged. ## Consequences diff --git a/docs/architecture/decisions/011-compiled-read-plan-for-packed-mode.md b/docs/architecture/decisions/011-compiled-read-plan-for-packed-mode.md index c2f1a8b..ca20649 100644 --- a/docs/architecture/decisions/011-compiled-read-plan-for-packed-mode.md +++ b/docs/architecture/decisions/011-compiled-read-plan-for-packed-mode.md @@ -2,7 +2,8 @@ ## Status -Accepted. Closes review #004 H1 + M1 (packed side) + L1 + L2; +Accepted. Implemented in 0.3.0 (phases 1–2, 2026-09-02). Closes review +#004 H1 + M1 (packed side) + L1 + L2; retires the "re-parse on demand" framing from ADR-007. A derisking POC on branch `readplan-poc` confirmed the `ReadPlan` shape covers every `BastType` arm in the current read loop before implementation diff --git a/docs/architecture/decisions/012-plan-fingerprinting-and-m1-closure.md b/docs/architecture/decisions/012-plan-fingerprinting-and-m1-closure.md index 2bee8a5..a78f2b8 100644 --- a/docs/architecture/decisions/012-plan-fingerprinting-and-m1-closure.md +++ b/docs/architecture/decisions/012-plan-fingerprinting-and-m1-closure.md @@ -2,8 +2,10 @@ ## Status -Accepted. Implemented (§3's `ValidationPlan`, 0.3.0 phase 7, 2026-08-31); -bundles three pieces of work into the 0.3.0 release so the +Accepted. Implemented in 0.3.0 — §3's `ValidationPlan` (phase 7, +2026-08-31), §1's fingerprinting (phase 6), §2a's owned `BastDoc` +(phases 3–4), §2b's `LeafMeta` (phase 5); all shipped 2026-09-02. +Bundles three pieces of work into the 0.3.0 release so the crate ships one round of breaking changes, not two (or three). The three pieces: (a) fingerprinting `ReadPlan`/`OffsetMap`, (b) closing the deferred M1 sites via an owned `BastDoc` + `OffsetMap` `LeafMeta`, diff --git a/docs/architecture/layout-engine.md b/docs/architecture/layout-engine.md index 960ad91..e5083ec 100644 --- a/docs/architecture/layout-engine.md +++ b/docs/architecture/layout-engine.md @@ -25,7 +25,7 @@ protocols. **Components:** - **`LayoutBuilder`** — constructed via `LayoutBuilder::new(bast_doc, root_name)` (requires a `struct` at the root), then `builder.build(&var_sizes) -> Result` where `var_sizes: &HashMap` maps variable-length field paths (and TUnion discriminator/variant keys) to their actual byte sizes. Used at write time when the consumer knows the data sizes upfront. The builder computes positions only; the consumer writes data via the [`data_access`](data-access.md) functions at the computed positions. -- **`SequentialReader`** — constructed via `SequentialReader::new(bast_doc, root_name)`, then driven by `reader.read_next(&buffer) -> Result, AlkTypeError>` until `Ok(None)`, or `reader.read_field(&buffer, path)` to seek a single field (which walks all preceding fields to reach the target). `reader.reset()` rewinds to the start. Used at read time when the consumer is parsing an incoming frame. +- **`SequentialReader`** — constructed via `engine.sequential_reader()` (shares the engine's compiled `ReadPlan` via `Arc` — see [ADR-011](decisions/011-compiled-read-plan-for-packed-mode.md)), then driven by `reader.read_next(&buffer) -> Result, AlkTypeError>` until `Ok(None)`, or `reader.read_field(&buffer, path)` to seek a single field (which walks all preceding fields to reach the target). `reader.reset()` rewinds to the start. Used at read time when the consumer is parsing an incoming frame. **How it works:** @@ -69,7 +69,7 @@ and safetensors. **Component:** -- **`OffsetMap`** — constructed via `OffsetMap::compute(&doc) -> Result` (requires a `struct` at the root). Walks the BAST typed tree once, computes fixed byte positions for each field based on type sizes and alignment. The output is a flat table of `(field_path, byte_range)` pairs (see [Public Types](#public-types)). Used for both read and write at known offsets. +- **`OffsetMap`** — constructed via `OffsetMap::compute(&doc) -> Result` (requires a `struct` at the root). Walks the BAST typed tree once, computes fixed byte positions for each field based on type sizes and alignment, and resolves each leaf's `LeafMeta` (kind, encoding, effective endianness — ADR-012 §2b). The output is a flat table of `(field_path, OffsetEntry)` pairs (see [Public Types](#public-types)). Used for both read and write at known offsets. **How it works:** @@ -312,22 +312,46 @@ discriminators, the discriminator is recorded under the synthetic path (schema `properties` order, with nested struct fields appearing inline under their parent's path prefix). +### `LeafMeta` / `OffsetEntry` (aligned mode, 0.3.0) + +```rust +pub struct LeafMeta { + pub kind: AlkTypeKind, + pub encoding: VariableEncoding, + pub endian: Endian, +} + +pub struct OffsetEntry { + pub range: ByteRange, + pub meta: LeafMeta, +} +``` + +`OffsetMap::compute` resolves each leaf's read/write metadata (kind, +variable-length encoding, effective endianness — field override else +container default, propagated the aligned-materializer way) alongside +its byte range, so `read_field`/`write_field` dispatch on the entry +without re-walking the BAST tree per access (ADR-012 §2b). + ### `OffsetMap` (aligned mode) -A flat table of `(field_path, byte_range)` pairs computed from a schema. +A flat table of `(field_path, OffsetEntry)` pairs computed from a schema. ```rust impl OffsetMap { - pub fn compute<'a>(doc: &'a BastDoc<'a>) -> Result; - pub fn get(&self, field_path: &str) -> Option<&ByteRange>; + pub fn compute(doc: &BastDoc) -> Result; + pub fn get(&self, field_path: &str) -> Option<&OffsetEntry>; pub fn total_size(&self) -> usize; - pub fn iter(&self) -> impl Iterator; + pub fn iter(&self) -> impl Iterator; + pub fn fingerprint(&self) -> u64; } ``` `compute` requires a `struct` at the root. `total_size` includes trailing alignment padding. `iter` yields fields in the BAST -`fields` array order (nested struct fields appearing inline). +`fields` array order (nested struct fields appearing inline). The map +carries `Hash + Eq` (ADR-012 §1); `fingerprint()` is the +stable-within-version hash for caching and schema handshakes. ## Design Decisions diff --git a/docs/plans/030-compiled-forms.md b/docs/plans/030-compiled-forms.md index 9ee563f..33ededd 100644 --- a/docs/plans/030-compiled-forms.md +++ b/docs/plans/030-compiled-forms.md @@ -1,5 +1,5 @@ --- -status: in-progress +status: done created: 2026-08-19 last_updated: 2026-09-02 adr: ADR-011, ADR-012 @@ -833,7 +833,34 @@ bench alongside `wire_vs_bast` to confirm the validation half of --- -## Phase 8 — Public API bump, docs, verification (ADR-011 step 6, ADR-012) +## Phase 8 — Public API bump, docs, verification (ADR-011 step 6, ADR-012) — **DONE (2026-09-02)** + +> **Status: implemented.** Version flipped 0.2.0 → 0.3.0; `lib.rs` +> re-exports complete (`ReadPlan` + sub-types, `LeafMeta`, +> `OffsetEntry`, `ValidationPlan` + sub-types from earlier phases). +> Docs: ADR-007 "Cost" section rewritten to the `Arc` cost +> (15.7 ns) with the old framing as a historical note (review #004 L2 +> closed); ADR-011/012 status blocks flipped to implemented; the +> architecture README ADR table rows updated; `layout-engine.md` +> rewritten for the 0.3.0 surface (`SequentialReader` construction via +> the engine factory, `OffsetMap` `OffsetEntry`/`LeafMeta`/`fingerprint` +> public-types section, `OffsetMap::compute(&BastDoc)` owned signature); +> `SequentialReader` module doc now points at the engine factory. +> Reviews #004 and #005 status flipped to closed. Bench (alktty +> `wire_vs_bast`, re-run on the 0.3.0 tree): read p64 98 ns/chunk +> (hand-rolled 5.7 µs/stream — parity held from phase 2), read p4k +> unchanged, `alktype_layout_build` **180 ns** (was ~1.2 µs — the +> phase-4 owned-doc cache removed the per-build re-parse, ~7x), +> `sequential_reader_new` 15.7 ns (unchanged), write p64 −3% +> (37.9 µs), `engine_compile` 590 µs (unchanged; dominated by +> meta-schema validation). No `validate_bytes`-stream bench was added: +> the phase-7 spot check (~0.2 µs/call plan-validate vs ~0.6 µs +> compile-per-call) stands as the validation-half measurement; a +> dedicated bench remains a follow-up if `alkcall` profiling motivates +> it. Downstream: `alktty` compiles against the path dep unchanged +> (the bench uses `LayoutBuilder::new`/`build` and +> `engine.sequential_reader()` — no touched signatures); +> `alkcall` has no dependency yet. **Goal:** Flip the version to 0.3.0, update `lib.rs` re-exports, update the architecture docs (ADR-007 "Cost" rewrite, ADR-011/012 status flip diff --git a/docs/reviews/004-performance-review.md b/docs/reviews/004-performance-review.md index c898ecc..6fd4661 100644 --- a/docs/reviews/004-performance-review.md +++ b/docs/reviews/004-performance-review.md @@ -1,6 +1,6 @@ --- -status: open -last_updated: 2026-08-17 +status: closed +last_updated: 2026-09-02 reviewed_artifacts: - src/sequential_reader.rs - src/bast.rs diff --git a/docs/reviews/005-plan-review-030.md b/docs/reviews/005-plan-review-030.md index 208f820..4fd53ec 100644 --- a/docs/reviews/005-plan-review-030.md +++ b/docs/reviews/005-plan-review-030.md @@ -1,6 +1,6 @@ --- -status: open -last_updated: 2026-08-20 +status: closed +last_updated: 2026-09-02 resolved_findings: 2026-08-20 (all 11 — see "Resolution" at the end) reviewed_artifacts: - docs/plans/030-compiled-forms.md diff --git a/src/sequential_reader.rs b/src/sequential_reader.rs index acb5371..484b5b6 100644 --- a/src/sequential_reader.rs +++ b/src/sequential_reader.rs @@ -119,7 +119,8 @@ pub enum FieldValue<'a> { /// fields 0..N-1 first. This is inherent to packed layouts where /// variable-length fields shift subsequent fields. /// -/// Construct with [`SequentialReader::new`], then drive with +/// Construct via [`crate::AlkTypeEngine::sequential_reader`] (shares +/// the engine's compiled plan), then drive with /// [`SequentialReader::read_next`] until it returns `Ok(None)`. Use /// [`SequentialReader::reset`] to walk the same buffer again, or /// [`SequentialReader::read_field`] to seek a single field by name