230345a867063d8b49d5d47b36249d80f9c66947
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 deepseek-v4-pro
|
230345a867 |
Validate BAST documents against the meta-schema at compile time
Resolves review #003 finding M3. - Add bast_meta::validate_bast_doc and call it from AlkTypeEngine::compile before parsing. Malformed annotations (unknown endian/encoding strings, non-integer align/maxLength, missing required properties) now surface as AlkTypeError::Schema instead of being silently tolerated by the parser. - Align the meta-schema TypeRef with the parser: allow inline struct/union/ enum as TypeRefs (the parser and builder already accepted them; the meta-schema and spec did not). Update bast-format.md TypeRef table and the BastType doc comment to the seven-form vocabulary. Verification: cargo test --release (410 pass), cargo clippy --all-targets -- -D warnings (clean), cargo doc --no-deps (clean), cargo build --target wasm32-unknown-unknown --release (clean). |
||
|
deepseek-v4-pro
|
e5c7cc1ca2 |
Fix offset-indirect, field-level endian, and aligned materialization
Resolves review #003 findings M1, M2, L1, and L2. - offset-indirect (L1 + M2 arm): rework read_*_indirect to same-buffer absolute offsets (safetensors-style, per the metatensor model) and add write_*_indirect. Wire the encoding dispatch into engine.read_field/ write_field and the aligned materializer. Previously the encoding was laid out but never read back. - field-level endian override (M1): thread field.effective_endian through sequential_reader, engine.read_field/write_field, and materialize_struct_aligned. Previously a per-field endian override was silently ignored, misreading multi-byte values. - aligned-mode materialization (M2): materialize fixed-size arrays via their vals[i] offset-map entries and maxLength reservations as zero-padded fixed-size slices (trailing NULs trimmed). Previously both read garbage or errored. - dead endian param (L2): drop the ignored endian argument from materialize_packed/materialize_aligned; endianness is read from the root struct. Verification: cargo test --release (404 pass), cargo clippy --all-targets -- -D warnings (clean), cargo build --target wasm32-unknown-unknown --release (clean), cargo llvm-cov --release (89.60% lines). |
||
 glm-5.2
|
54fd112fde |
Remove v0.1.0 custom-keyword machinery (step 8)
The BAST parser (step 3) and BAST-native validator (step 5) replaced the v0.1.0 custom-keyword accessor layer; step 7 moved the builder to BAST output. This step removes the now-dead code: Removed from src/schema.rs: - get_alktype_kind / get_alktype_kind_enum / get_alktype_kind_loose / get_alktype_kind_loose_enum (replaced by the BAST parser's kind dispatch) - normalize_refs / inline_union_variant_refs + helpers (BAST refs are always #/$defs/<name>; resolution is a single hash lookup, variant refs resolve lazily) - parse_encoding / parse_align / parse_max_length / parse_endian (bast.rs has its own BAST-property-form copies) - parse_discriminator + DiscriminatorKind (replaced by bast::BastDiscriminator; builder has its own Discriminator enum) - resolve_ref / resolve_ref_or_inline (replaced by BastDoc::lookup_def / resolve_typeref) - FromStr impl, as_str, Endian::from_schema, ALKTYPE_PREFIX, BYTE_DISCRIMINATOR_TYPES, and the associated unit tests Kept: AlkTypeKind enum + methods (type_size, natural_alignment, is_fixed_size, needs_endian, is_composite, is_variable_length, to_bast_str, from_bast_str), Display (now backed by to_bast_str), Endian, VariableEncoding, U32_SIZE, DISCRIMINATOR_PATH. src/lib.rs: dropped the 13 schema::* helper re-exports and DiscriminatorKind from the public surface; kept Endian, AlkTypeKind, VariableEncoding. Doc/comment updates: bast.rs, builder.rs, engine.rs, error.rs — removed references to the deleted functions and the AlkType:* keyword form. The jsonschema crate remains a dependency (validate_json path + BAST meta-schema validation); build_validator was already repurposed in step 6 (no custom keywords). Verification: - cargo test --release: 389 pass (312 lib + 77 integration) - cargo clippy --all-targets -- -D warnings: clean - cargo doc --no-deps: clean - cargo build --target wasm32-unknown-unknown --release: clean |
||
|
glm-5.2
|
ba7f8e1bad |
validate_json against consumer-provided JSON Schema (step 6)
- AlkTypeEngine::compile gains a 4th param json_schema: Option<&Value>. When Some, a standard jsonschema::Validator is built from the consumer-provided JSON Schema and stored for the JSON-validation path. When None, validate_json returns AlkTypeError::Schema and is_valid_json returns false (D-BAST-007). - validate_json / is_valid_json signatures unchanged (per semver contract). Behavior: they now validate against the consumer JSON Schema, not a custom-keyword validator built from the alktype schema. The BAST document is not involved in this path. - build_validator repurposed (deferred decision #2): same signature, now builds a standard jsonschema::Validator with no custom keywords. Behavioral break, not a type break. Re-export kept. - Removed the 19 jsonschema::Keyword implementations and the 4 define_*_validator! macros (dead on the bytes path since step 5, now dead on the JSON path too). The is_rfc3339_timestamp helper lives on in bast_validation.rs (already copied there in step 5). - All compile call sites updated to pass None for json_schema (the layout/read/write/validate_bytes tests don't need JSON validation). - New tests: validate_json accepts/rejects against consumer JSON Schema, returns Schema error when no JSON Schema supplied, is_valid_json false when no schema, independence from BAST doc, malformed JSON Schema build error, nested object JSON Schema. Verification: - cargo test --release: 409 pass (332 lib + 77 integration) - cargo clippy --all-targets -- -D warnings: clean - cargo build --target wasm32-unknown-unknown --release: clean - cargo doc --no-deps: clean |
||
|
glm-5.2
|
f853dafaf1 |
Add BAST-native validator for validate_bytes (step 5)
Replace the jsonschema custom-keyword validator on the bytes path with a recursive walker over the BAST typed tree. The materializer already guarantees structural correctness; the validator enforces only the value-domain constraints expressed in the BAST document. - New `src/bast_validation.rs`: `validate_value` dispatches on `BastType`, resolving `$ref`s lazily via `BastDoc::resolve_typeref`. Constraint arms: integer ranges (Int8..Uint64), float finiteness, string/bytes `maxLength` (from `BastField::max_length`), RFC 3339 timestamp shape, enum index bounds, union variant dispatch (recurses into the variant, recovering OQ-008 per-variant constraints), struct field presence, array count, record values. - `engine::validate_bytes` now calls `bast_validation::validate_value` instead of `self.validator.validate`. The `validator` field is still built and used by `validate_json`/`is_valid_json` (step 6 reworks those). - Enum index bounds check (`idx < values.len()`) fixes the v0.1.0 dead constraint: the built-in `enum` keyword checked string membership, but the materializer emits a numeric index that never matched. - Errors constructed via `jsonschema::ValidationError::custom` so `AlkTypeError::Validation` keeps its payload type uniform with the `validate_json` path (D-BAST-009). - `lib.rs`: add `pub mod bast_validation;` (engine-internal, not re-exported in the `pub use` block) and update the module doc. Verification: - cargo test --release: 446 pass (369 lib + 77 integration) - cargo clippy --all-targets -- -D warnings: clean - cargo build --target wasm32-unknown-unknown --release: clean - cargo doc --no-deps: clean |
||
|
glm-5.2
|
04573e1d86 |
Wire compile() to BAST document + root name (step 4)
Step 4 of the BAST pivot: the layout engines, materializer, tunion
dispatch, and engine now consume the BAST typed tree (BastDoc/
BastStruct/BastField/BastType/...) instead of walking raw JSON with
get_alktype_kind*.
Breaking changes (per the pivot plan's semver contract):
- AlkTypeEngine::compile signature:
compile(schema: &mut Value, mode)
-> compile(bast_doc: &Value, root_name: &str, mode)
Drops &mut (BAST needs no in-place normalize_refs); adds required
root_name (D-BAST-001); input is a BAST document, not a custom-keyword
JSON Schema.
- OffsetMap::compute, LayoutBuilder::new, SequentialReader::new now take
a BAST document (&Value) + root_name (or &BastDoc) instead of a
v0.1.0 schema.
- tunion::read_byte_discriminator / read_field_discriminator /
resolve_variant / discriminator_size now take &BastUnion instead of
&Value.
- materialize::materialize_packed / materialize_aligned now take
&BastDoc instead of &Value.
Key design points:
- The engine stores a clone of the BAST Value + root_name so
sequential_reader() and read_field() can re-parse the typed tree on
demand without lifetime entanglement with the caller's Value.
- resolution is a single hash lookup via BastDoc::resolve_typeref;
no normalize_refs, no inline_union_variant_refs.
- bast.rs gains BastField::synthetic() (pub(crate)) for constructing
synthetic fields wrapping TypeRefs (array elements, record values,
union variants — these aren't fields and carry no field annotations).
- The v0.1.0 schema.rs helpers and validation.rs custom-keyword
validators remain defined (step 8 removes them). build_validator still
runs on the BAST doc — with no AlkType:* keywords present, the custom
factories don't trigger and jsonschema performs structural validation
only. The validate_bytes value-constraint enforcement (maxLength, enum
bounds) is step 5's concern (the BAST-native validator).
Tests:
- All engine, layout, materialize, tunion, and integration tests
converted to BAST format (kind/fields vocabulary, /
composition). Expected validation outcomes for the layout path are
identical; the maxLength/enum-bounds validate_bytes tests are step 5's
regression target.
- builder.rs::builder_chunk_header_compiles_in_packed_mode uses a
hand-written BAST doc (the builder still emits v0.1.0 format; step 7
converts it).
Verification:
- cargo test --release: 425 pass (348 lib + 77 integration)
- cargo clippy --all-targets -- -D warnings: clean
- cargo build --target wasm32-unknown-unknown --release: clean
- cargo doc --no-deps: clean
|
||
|
glm-5.2
|
c0217d91a8 |
Resolve v0.1.0 open questions and fix production-readiness issues
POC: /workspace/alktype-builder-poc/ (18/18 tests pass, findings in
docs/research/alktype-builder-poc/findings.md). Round 2 adds the SFTP
Packet validate_bytes tests (7 new: valid Init/Read/Write/Status,
short buffer, unknown discriminator, over-maxLength Bytes).
Open questions resolved (OQ-004 through OQ-008):
- OQ-004: Discriminator::Field name is String (already implemented;
docs updated to mark resolved)
- OQ-005: Both union discriminator kinds return the same shape:
{__discriminator, ...variant-fields}. Field-name path also had a
real offset bug (returned start, not end) - fixed.
- OQ-006: builder.md Example 3 now wraps the Union in a
Schema::struct_().field("payload", ...) and merges $defs via
Definitions::merge_into (matches the engine's AlkType:Struct-at-root
constraint and the SFTP wire shape)
- OQ-007: Bytes materialization is array-of-u8 (Value::Array of
Value::Number, one entry per byte 0..=255). BytesValidator accepts
both Value::String (validate_json) and Value::Array (validate_bytes).
maxLength = max byte count. Replaces the lossy from_utf8_lossy path
that corrupted non-UTF-8 bytes and broke maxLength semantics.
- OQ-008 (new): UnionValidator now dispatches to variant schemas via
sub-validators built at factory time. AlkTypeEngine::compile calls
schema::inline_union_variant_refs before build_validator to inline
$refs in union mapping entries (necessary because union_factory
receives the union node, but $defs live at the schema root).
Production-readiness fixes in src/ (no stubs/hedges in a published crate):
- materialize.rs: Record stub -> full count-prefixed key/value pair
implementation per schema-layer.md TRecord
- materialize.rs: root_of() was broken (returned the current node, not
the schema root) -> root schema threaded through every recursive call
so resolve_ref_or_inline can resolve $refs for nested composites
- builder.rs: LengthPrefixed encoding setter was a no-op when the
keyword was already in object form -> complete the branch (updates
the encoding entry in place for both LengthPrefixed and OffsetIndirect)
- builder.rs, engine.rs: POC-referencing comments cleaned up; the
round-trip test's or_else fallback (papering over write_field being
aligned-only) replaced with direct byte writes
Documentation:
- builder.md: Example 3 updated; Discriminator::Field spec shows String;
Open Questions section updated (OQ-004 resolved)
- validation.md: AlkType:Bytes and AlkType:Union validator descriptions
updated for array-of-u8 form and variant dispatch
- open-questions.md: OQ-004/005/006/007/008 marked resolved; new
Validation theme entries
- questions/004-008: individual OQ files updated with resolutions
- findings.md: round 2 results documented
Verification:
- cargo test: 369 -> 391 tests pass (22 new: 14 materialize, 5
inline_union_variant_refs, 3 validation/builder)
- cargo clippy --all-targets: clean
- POC: 11 -> 18 tests (7 new SFTP Packet tests); all pass
|
||
|
glm-5.2
|
5588278451 |
Implement builder (ADR-009) and validate_bytes (ADR-010) for v0.1.0
POC: /workspace/alktype-builder-poc/ (11/11 tests pass, findings in docs/research/alktype-builder-poc/findings.md). The POC code lives outside the repo per the internal dev convention. Implementation: - src/builder.rs: Schema, Definitions, Discriminator types. Constructors for all 19 AlkType kinds + standard JSON Schema types (object/array/ string/integer/number/boolean/null/any). Setters for ADR-003 annotations (endian/align/encoding/max_length), composite builders (field/required/ items/mapping), and standard JSON Schema constraints (minimum/maximum/ minLength/minItems/maxItems/format/title/description). 16 unit tests. - src/materialize.rs: materialize_packed and materialize_aligned functions that walk a schema + buffer to produce a serde_json::Value tree. Recurses into Struct, Array, Union (byte-offset discriminator). Record is stubbed (deferred for the POC scope). - src/engine.rs: AlkTypeEngine::validate_bytes(&[u8]) added (ADR-010). Dispatches on layout mode, materializes Value, then validates against the existing jsonschema validator. 7 unit tests. - src/lib.rs: pub mod builder, pub mod materialize; re-exports Schema, Definitions, Discriminator. Verification: - cargo test: 346 -> 369 tests pass (23 new: 16 builder, 7 validate_bytes) - cargo clippy --all-targets -- -D warnings: clean - POC (11 tests): builder round-trip + validate_bytes (packed + aligned) + validate_json for call payloads; all pass Findings: - Top-level schema must be AlkType:Struct (existing constraint); unions are field types within a struct. Builder spec Example 3 needs a doc fix. - Builder field order preserved (preserve_order feature, load-bearing for packed mode). - validate_bytes correctly distinguishes Access (read phase) from Validation (validate phase) errors, with field paths. - validate_json path unchanged for call payloads. Open questions surfaced (OQ-005, OQ-006, OQ-007) tracked in findings.md; to resolve before the SFTP Packet POC round. |
||
|
glm-5.2
|
57d8ed25ba |
Add tests for coverage gaps S1, S2, S3, S5, S6, S7 (285→346 tests, 88.9%→91.9% lines)
- S1 (error.rs): 5 tests for Display impl on all 4 AlkTypeError variants + Error::source(). error.rs 0%→100%. - S2 (validation.rs, macros.rs): 28 tests for validate() (Result-returning) method on every validator + factory rejection arms for all 12 keywords. validation.rs 85%→97.5% lines / 100% fns; macros.rs 75.7%→93.1% / 100% fns. - S3 (engine.rs): 3 tests for read_field on nested struct leaf fields, Bytes, and Timestamp. engine.rs fns 95%→95.3%. - S5 (layout_builder.rs): 2 tests for union 'variant must be Struct' error (byte + field discriminator). - S6 (schema.rs): 6 tests for as_str() round-trip (all 19 kinds), Display, needs_endian(), is_composite(), get_alktype_kind_enum(). schema.rs 91.2%→97.5% lines / 94.3% fns. - S7 (offset_map.rs): 1 test for nested-struct-without-properties schema error. Updated docs/reviews/001-coverage-analysis.md with resolution section and partial-resolved status. Remaining: S4 (sequential_reader error paths, medium effort) and S8 (overflow guards, recommended to skip for v1). |
||
|
glm-5.2
|
5e268e8f47 |
Rebrand TypeDef to AlkType in code, keyword strings, and docs
Rename all 19 JSON Schema custom keyword strings from "TypeDef:*"
to "AlkType:*" (e.g., "TypeDef:Struct" -> "AlkType:Struct")
across source, tests, and docs. This is a breaking change to the
schema format itself — existing schemas using the old keywords
must be updated.
Rename the Rust identifiers:
- TypedefEngine -> AlkTypeEngine
- TypedefError -> AlkTypeError
- TypeDefKind -> AlkTypeKind
- TYPEDEF_PREFIX -> ALKTYPE_PREFIX
- get_typedef_kind{,_loose,_loose_enum,_enum} ->
get_alktype_kind{,_loose,_loose_enum,_enum}
Update error message strings ("unknown TypeDef kind" ->
"unknown AlkType kind"), 11 test function names containing
typedef_kind/to_typedef_error, and doc-comment prose ("TypeDef
kind" -> "AlkType kind", "typedef engine" -> "alktype
engine", "typedef schema" -> "alktype schema"). Fix the broken
docs/architecture/crates/typedef/ path references in source doc
comments to point at docs/architecture/ directly. Rebrand the
typedef:annotation test fixture and the "not-a-typedef" test
string to their alktype equivalents.
Update ~20 generic "typedef" prose references in the architecture
docs ("typedef is the binary struct engine", "use typedef",
"typedef limitation", "replaced by typedef", etc.) to alktype.
Rename TypedefEngine in the ADR-007 code example to AlkTypeEngine.
Preserve as provenance per the prior prose-rebrand decision:
typedef.ts references (external TypeBox source file),
docs/research/alknet-typedef/findings.md research citations,
/workspace/alknet-typedef-poc/ POC path, and the
"alknet-typedef:" research section headers in findings.
Build, 295 tests, and clippy all pass clean.
|
||
|
glm-5.2
|
2c4a4994dc |
Port alknet-typedef crate from alknet
Copy the binary struct engine (src/, tests/) verbatim from alknet/crates/alknet-typedef and create a standalone Cargo.toml (workspace-inherited fields inlined). Port the architecture docs (specs, ADRs 095-102, OQs 069-071) from alknet's nested multi-crate layout to a flat single-crate layout, fixing relative link paths. Build, 295 tests, and clippy all pass clean. |