e5f1b9d8256577d3213e9aac6080de547afd898f
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 glm-5.3-flash
|
e5f1b9d825 |
Wire packed read path through ReadPlan (ADR-011 steps 2-4, plan phase 2)
SequentialReader now walks Arc<ReadPlan> instead of re-parsing the BAST typed tree per field (the 400x read-path gap, review #004 H1); materialize_packed walks the same plan, unifying the two packed read-side consumers on one compiled form. - SequentialReader::new(Arc<ReadPlan>) -> Self, infallible: the fallible BastDoc parse moved to ReadPlan::compile (phase 1). The reader holds the plan Arc + cursor state only; schema() returns the Arc<Value> retained on the plan (review #005 H2 — no self-referential struct); new plan() accessor exposes the shared plan. - ReadPlan carries schema: Arc<Value> (set at compile; sub-plans hold a Null placeholder — only the root plan is handed out). - materialize_packed(&ReadPlan, &[u8]): plan-walking packed materializer. The aligned path keeps walking BastDoc with the retained dummy_field_for/ty_source/materialize_typeref_packed helpers (phase 5 Scope Boundary: aligned structure walk is the permanent 0.3.0 design). - Engine: Layout::Packed stores Arc<ReadPlan> alongside the builder; sequential_reader() is an Arc::clone (was a full-document Value clone); packed validate_bytes calls materialize_packed(&self.plan). - Stride (deferred decision 4): FieldValue::Array now reports the true stride for fixed-size struct/nested-array elements (0.2.0 returned 0); doc comment documents the behavioral change; no existing test asserted the 0, so none needed changing. - Two parity subtleties found and preserved: (a) materialize_plan_composite unwraps the plan's anonymous single-field wrapper for primitive array elements/record values — without it, materialized records nest each leaf under a synthetic object (caught by the record parity test); (b) field-disc unions keep 0.2.0's materialized key order (__discriminator first), observable under preserve_order. Both are now covered by plan-phase tests or construction. Bench (alktty wire_vs_bast, 1024 chunks/stream): packed read 2.27 us/chunk (review #004) -> 98 ns/chunk p64 / 100 ns/chunk p4k (~23x; the 400x gap closes to ~17x vs hand-rolled 5.6 ns/chunk). Residual gap is the per-field String allocation mandated by the unchanged (String, FieldValue) read_next signature (2 allocs/chunk) plus data_access bounds checks. sequential_reader() construction: 15.7 ns (was a whole-document clone). Verification: 465 tests pass unchanged (the existing reader/ materialize/engine suites drive the rewrite through the public API — only constructor call sites moved to ReadPlan::compile); clippy -D warnings clean; cargo doc zero warnings; wasm32 release build green. |
||
 deepseek-v4-pro
|
510553d800 |
Remove the Timestamp kind
The Timestamp kind was a residual from an early research reference. It was byte-identical to String everywhere (length-prefixed UTF-8) and its only distinguishing behavior was a hand-rolled non-strict RFC 3339 check that the docs admitted was incomplete (Feb 31 passes, seconds range unchecked, no leap seconds). JSON-level timestamp validation is jsonschema's job (format: date-time on the validate_json path), not alktype's. Removes the AlkTypeKind::Timestamp variant, its to_bast_str/from_bast_str mapping, the builder's Schema::timestamp() constructor, the validate_timestamp/is_rfc3339_timestamp validator arms, and the materializer/reader/engine timestamp arms. Updates the meta-schema primitive enum (14 -> 13), the spec docs (bast-format.md, schema-layer.md, builder.md, validation.md, overview.md, data-access.md, README.md), and the kind-count references (19 -> 18). Verification: cargo test --release (407 pass), cargo clippy --all-targets -- -D warnings (clean), cargo doc --no-deps (clean), cargo build --target wasm32-unknown-unknown --release (clean). |
||
|
deepseek-v4-pro
|
62ed009281 |
Resolve review #003 nits N1, N3, N4
- N1: number_from_f64 now returns AlkTypeError::Access for non-finite floats instead of silently materializing Value::Null. A NaN/Inf in the buffer surfaces as a clear access error rather than a misleading 'expected a number' validation error. - N3: drop the dead Value::String arm from check_bytes; the materializer only ever emits bytes as an array of u8. Update the validation-model docs to match. - N4: fix stale ADR references in doc comments (ADR-096 -> ADR-002, ADR-097 -> ADR-003, ADR-098 -> ADR-004, ADR-101 -> ADR-007). N2 (BastType::alk_kind returning Struct for ) left as documented; every current caller resolves the ref first, so forcing Option through the call sites is churn without benefit. Verification: cargo test --release (411 pass), cargo clippy --all-targets -- -D warnings (clean), cargo doc --no-deps (clean). |
||
|
deepseek-v4-pro
|
e5c7cc1ca2 |
Fix offset-indirect, field-level endian, and aligned materialization
Resolves review #003 findings M1, M2, L1, and L2. - offset-indirect (L1 + M2 arm): rework read_*_indirect to same-buffer absolute offsets (safetensors-style, per the metatensor model) and add write_*_indirect. Wire the encoding dispatch into engine.read_field/ write_field and the aligned materializer. Previously the encoding was laid out but never read back. - field-level endian override (M1): thread field.effective_endian through sequential_reader, engine.read_field/write_field, and materialize_struct_aligned. Previously a per-field endian override was silently ignored, misreading multi-byte values. - aligned-mode materialization (M2): materialize fixed-size arrays via their vals[i] offset-map entries and maxLength reservations as zero-padded fixed-size slices (trailing NULs trimmed). Previously both read garbage or errored. - dead endian param (L2): drop the ignored endian argument from materialize_packed/materialize_aligned; endianness is read from the root struct. Verification: cargo test --release (404 pass), cargo clippy --all-targets -- -D warnings (clean), cargo build --target wasm32-unknown-unknown --release (clean), cargo llvm-cov --release (89.60% lines). |
||
 glm-5.2
|
04573e1d86 |
Wire compile() to BAST document + root name (step 4)
Step 4 of the BAST pivot: the layout engines, materializer, tunion
dispatch, and engine now consume the BAST typed tree (BastDoc/
BastStruct/BastField/BastType/...) instead of walking raw JSON with
get_alktype_kind*.
Breaking changes (per the pivot plan's semver contract):
- AlkTypeEngine::compile signature:
compile(schema: &mut Value, mode)
-> compile(bast_doc: &Value, root_name: &str, mode)
Drops &mut (BAST needs no in-place normalize_refs); adds required
root_name (D-BAST-001); input is a BAST document, not a custom-keyword
JSON Schema.
- OffsetMap::compute, LayoutBuilder::new, SequentialReader::new now take
a BAST document (&Value) + root_name (or &BastDoc) instead of a
v0.1.0 schema.
- tunion::read_byte_discriminator / read_field_discriminator /
resolve_variant / discriminator_size now take &BastUnion instead of
&Value.
- materialize::materialize_packed / materialize_aligned now take
&BastDoc instead of &Value.
Key design points:
- The engine stores a clone of the BAST Value + root_name so
sequential_reader() and read_field() can re-parse the typed tree on
demand without lifetime entanglement with the caller's Value.
- resolution is a single hash lookup via BastDoc::resolve_typeref;
no normalize_refs, no inline_union_variant_refs.
- bast.rs gains BastField::synthetic() (pub(crate)) for constructing
synthetic fields wrapping TypeRefs (array elements, record values,
union variants — these aren't fields and carry no field annotations).
- The v0.1.0 schema.rs helpers and validation.rs custom-keyword
validators remain defined (step 8 removes them). build_validator still
runs on the BAST doc — with no AlkType:* keywords present, the custom
factories don't trigger and jsonschema performs structural validation
only. The validate_bytes value-constraint enforcement (maxLength, enum
bounds) is step 5's concern (the BAST-native validator).
Tests:
- All engine, layout, materialize, tunion, and integration tests
converted to BAST format (kind/fields vocabulary, /
composition). Expected validation outcomes for the layout path are
identical; the maxLength/enum-bounds validate_bytes tests are step 5's
regression target.
- builder.rs::builder_chunk_header_compiles_in_packed_mode uses a
hand-written BAST doc (the builder still emits v0.1.0 format; step 7
converts it).
Verification:
- cargo test --release: 425 pass (348 lib + 77 integration)
- cargo clippy --all-targets -- -D warnings: clean
- cargo build --target wasm32-unknown-unknown --release: clean
- cargo doc --no-deps: clean
|
||
|
glm-5.2
|
5f88bca0d8 |
Fix L2: replace unreachable! with Err for untrusted-schema safety
The immediate downstream consumer (alkcall) accepts schemas from
arbitrary internet peers in its hub/spoke topology. A panic is the
wrong failure mode for a malicious or unsupported schema - an Err
the caller can handle is correct.
Three sites converted:
- offset_map.rs: _ => Err(Offset { unsupported AlkType kind for
aligned offset computation })
- layout_builder.rs: _ => Err(Offset { unsupported AlkType kind
for packed layout computation })
- materialize.rs: _ => Err(Schema { internal: union discriminator
type N is not a supported byte discriminator })
The materialize.rs site uses Schema (not Access) because a wrong
disc_type is a schema-authoring bug (parse_discriminator should have
caught it), not a buffer-access error. The sibling sites in
sequential_reader.rs and tunion.rs already returned Err(Schema) -
only materialize.rs was the holdout.
Note: the k if k.is_fixed_size() guard in offset_map and
layout_builder means the compiler cannot enforce exhaustiveness at
compile time. Converting _ from unreachable! to Err is the runtime
mitigation. A future refactor could list all fixed-size kinds
explicitly to restore compile-time checking. Deferred to a separate
cleanup pass.
Verified: no unreachable! remains in production code (the one
remaining hit at offset_map.rs:688 is inside a #[test] fn).
cargo test --release: 396 tests pass, 0 failures
cargo clippy --all-targets -- -D warnings: clean
cargo build --target wasm32-unknown-unknown --release: clean (prior)
|
||
|
glm-5.2
|
a975befdd1 |
Fix M1, M2, L1, L3 from code review #002
Four of seven review findings resolved. 5 new tests (391 -> 396 crate tests; 438 -> 443 total). cargo test, clippy, wasm32 all green. M2 (data_access.rs): write_bytes now validates data_len fits in u32 before the length-prefix cast. A >4GiB blob returns Access error instead of silently writing a truncated length prefix (silent data corruption on read-back). M1 (builder.rs): Definitions::merge_into rewritten to access self.defs directly instead of round-tripping through self.build() with a double- cloned/unwrap_or_default chain that could silently drop definitions on a shape mismatch. 4 new tests: insert-when-absent, merge-into- existing, overwrite-duplicate-keys, no-op-on-non-object-top. L1 (materialize.rs): byte-offset discriminator arm of materialize_union_packed now uses checked_add for offset+disc_offset and disc_abs_offset+disc_size, returning Access error on overflow. Mirrors the existing sequential_reader.rs::read_union_value pattern. L3 (error.rs): AlkTypeError::source() now returns Some(inner) for the Validation variant (jsonschema::ValidationError implements std::error::Error). Existing source_returns_none_for_all_variants test split into source_returns_none_for_schema_offset_access and source_returns_some_for_validation_variant. Deferred: L2 (unreachable! -> Err, defense-in-depth), N1 (non-strict RFC 3339 validator, docs-only), N2 (FieldValue::Bytes for Record, API asymmetry). Review doc updated with resolution section. |
||
|
glm-5.2
|
c0217d91a8 |
Resolve v0.1.0 open questions and fix production-readiness issues
POC: /workspace/alktype-builder-poc/ (18/18 tests pass, findings in
docs/research/alktype-builder-poc/findings.md). Round 2 adds the SFTP
Packet validate_bytes tests (7 new: valid Init/Read/Write/Status,
short buffer, unknown discriminator, over-maxLength Bytes).
Open questions resolved (OQ-004 through OQ-008):
- OQ-004: Discriminator::Field name is String (already implemented;
docs updated to mark resolved)
- OQ-005: Both union discriminator kinds return the same shape:
{__discriminator, ...variant-fields}. Field-name path also had a
real offset bug (returned start, not end) - fixed.
- OQ-006: builder.md Example 3 now wraps the Union in a
Schema::struct_().field("payload", ...) and merges $defs via
Definitions::merge_into (matches the engine's AlkType:Struct-at-root
constraint and the SFTP wire shape)
- OQ-007: Bytes materialization is array-of-u8 (Value::Array of
Value::Number, one entry per byte 0..=255). BytesValidator accepts
both Value::String (validate_json) and Value::Array (validate_bytes).
maxLength = max byte count. Replaces the lossy from_utf8_lossy path
that corrupted non-UTF-8 bytes and broke maxLength semantics.
- OQ-008 (new): UnionValidator now dispatches to variant schemas via
sub-validators built at factory time. AlkTypeEngine::compile calls
schema::inline_union_variant_refs before build_validator to inline
$refs in union mapping entries (necessary because union_factory
receives the union node, but $defs live at the schema root).
Production-readiness fixes in src/ (no stubs/hedges in a published crate):
- materialize.rs: Record stub -> full count-prefixed key/value pair
implementation per schema-layer.md TRecord
- materialize.rs: root_of() was broken (returned the current node, not
the schema root) -> root schema threaded through every recursive call
so resolve_ref_or_inline can resolve $refs for nested composites
- builder.rs: LengthPrefixed encoding setter was a no-op when the
keyword was already in object form -> complete the branch (updates
the encoding entry in place for both LengthPrefixed and OffsetIndirect)
- builder.rs, engine.rs: POC-referencing comments cleaned up; the
round-trip test's or_else fallback (papering over write_field being
aligned-only) replaced with direct byte writes
Documentation:
- builder.md: Example 3 updated; Discriminator::Field spec shows String;
Open Questions section updated (OQ-004 resolved)
- validation.md: AlkType:Bytes and AlkType:Union validator descriptions
updated for array-of-u8 form and variant dispatch
- open-questions.md: OQ-004/005/006/007/008 marked resolved; new
Validation theme entries
- questions/004-008: individual OQ files updated with resolutions
- findings.md: round 2 results documented
Verification:
- cargo test: 369 -> 391 tests pass (22 new: 14 materialize, 5
inline_union_variant_refs, 3 validation/builder)
- cargo clippy --all-targets: clean
- POC: 11 -> 18 tests (7 new SFTP Packet tests); all pass
|
||
|
glm-5.2
|
5588278451 |
Implement builder (ADR-009) and validate_bytes (ADR-010) for v0.1.0
POC: /workspace/alktype-builder-poc/ (11/11 tests pass, findings in docs/research/alktype-builder-poc/findings.md). The POC code lives outside the repo per the internal dev convention. Implementation: - src/builder.rs: Schema, Definitions, Discriminator types. Constructors for all 19 AlkType kinds + standard JSON Schema types (object/array/ string/integer/number/boolean/null/any). Setters for ADR-003 annotations (endian/align/encoding/max_length), composite builders (field/required/ items/mapping), and standard JSON Schema constraints (minimum/maximum/ minLength/minItems/maxItems/format/title/description). 16 unit tests. - src/materialize.rs: materialize_packed and materialize_aligned functions that walk a schema + buffer to produce a serde_json::Value tree. Recurses into Struct, Array, Union (byte-offset discriminator). Record is stubbed (deferred for the POC scope). - src/engine.rs: AlkTypeEngine::validate_bytes(&[u8]) added (ADR-010). Dispatches on layout mode, materializes Value, then validates against the existing jsonschema validator. 7 unit tests. - src/lib.rs: pub mod builder, pub mod materialize; re-exports Schema, Definitions, Discriminator. Verification: - cargo test: 346 -> 369 tests pass (23 new: 16 builder, 7 validate_bytes) - cargo clippy --all-targets -- -D warnings: clean - POC (11 tests): builder round-trip + validate_bytes (packed + aligned) + validate_json for call payloads; all pass Findings: - Top-level schema must be AlkType:Struct (existing constraint); unions are field types within a struct. Builder spec Example 3 needs a doc fix. - Builder field order preserved (preserve_order feature, load-bearing for packed mode). - validate_bytes correctly distinguishes Access (read phase) from Validation (validate phase) errors, with field paths. - validate_json path unchanged for call payloads. Open questions surfaced (OQ-005, OQ-006, OQ-007) tracked in findings.md; to resolve before the SFTP Packet POC round. |