Resolve v0.1.0 open questions and fix production-readiness issues

POC: /workspace/alktype-builder-poc/ (18/18 tests pass, findings in
docs/research/alktype-builder-poc/findings.md). Round 2 adds the SFTP
Packet validate_bytes tests (7 new: valid Init/Read/Write/Status,
short buffer, unknown discriminator, over-maxLength Bytes).

Open questions resolved (OQ-004 through OQ-008):
- OQ-004: Discriminator::Field name is String (already implemented;
  docs updated to mark resolved)
- OQ-005: Both union discriminator kinds return the same shape:
  {__discriminator, ...variant-fields}. Field-name path also had a
  real offset bug (returned start, not end) - fixed.
- OQ-006: builder.md Example 3 now wraps the Union in a
  Schema::struct_().field("payload", ...) and merges $defs via
  Definitions::merge_into (matches the engine's AlkType:Struct-at-root
  constraint and the SFTP wire shape)
- OQ-007: Bytes materialization is array-of-u8 (Value::Array of
  Value::Number, one entry per byte 0..=255). BytesValidator accepts
  both Value::String (validate_json) and Value::Array (validate_bytes).
  maxLength = max byte count. Replaces the lossy from_utf8_lossy path
  that corrupted non-UTF-8 bytes and broke maxLength semantics.
- OQ-008 (new): UnionValidator now dispatches to variant schemas via
  sub-validators built at factory time. AlkTypeEngine::compile calls
  schema::inline_union_variant_refs before build_validator to inline
  $refs in union mapping entries (necessary because union_factory
  receives the union node, but $defs live at the schema root).

Production-readiness fixes in src/ (no stubs/hedges in a published crate):
- materialize.rs: Record stub -> full count-prefixed key/value pair
  implementation per schema-layer.md TRecord
- materialize.rs: root_of() was broken (returned the current node, not
  the schema root) -> root schema threaded through every recursive call
  so resolve_ref_or_inline can resolve $refs for nested composites
- builder.rs: LengthPrefixed encoding setter was a no-op when the
  keyword was already in object form -> complete the branch (updates
  the encoding entry in place for both LengthPrefixed and OffsetIndirect)
- builder.rs, engine.rs: POC-referencing comments cleaned up; the
  round-trip test's or_else fallback (papering over write_field being
  aligned-only) replaced with direct byte writes

Documentation:
- builder.md: Example 3 updated; Discriminator::Field spec shows String;
  Open Questions section updated (OQ-004 resolved)
- validation.md: AlkType:Bytes and AlkType:Union validator descriptions
  updated for array-of-u8 form and variant dispatch
- open-questions.md: OQ-004/005/006/007/008 marked resolved; new
  Validation theme entries
- questions/004-008: individual OQ files updated with resolutions
- findings.md: round 2 results documented

Verification:
- cargo test: 369 -> 391 tests pass (22 new: 14 materialize, 5
  inline_union_variant_refs, 3 validation/builder)
- cargo clippy --all-targets: clean
- POC: 11 -> 18 tests (7 new SFTP Packet tests); all pass
This commit is contained in:
glm-5.2 committed 2026-08-11 07:28:24 +00:00
1 parent c6893eece8
commit c0217d91a8
15 files changed
+1553 -493

No files matched your search

+76 -13
View File
@@ -243,9 +243,14 @@ impl Schema {
/// Variable-length encoding strategy.
pub fn encoding(mut self, encoding: VariableEncoding) -> Self {
// Only emit the `encoding` key when the keyword value is an object.
// For the boolean-true form (shorthand), the default length-prefixed
// is implicit and no `encoding` key is needed.
// The `encoding` annotation lives inside the AlkType:* keyword's
// value object (see schema-layer.md §"Variable-length encoding"):
// { "AlkType:String": { "encoding": "offset-indirect" } }
// For the boolean-true shorthand (`{ "AlkType:String": true }`),
// `length-prefixed` is the implicit default and no `encoding` key
// is emitted. `offset-indirect` always requires the object form.
// If the keyword value is already an object (e.g. from a prior
// setter), update its `encoding` entry in place.
let kind_key = self
.object
.keys()
@@ -254,19 +259,40 @@ impl Schema {
if let Some(k) = kind_key {
match encoding {
VariableEncoding::LengthPrefixed => {
// Default — make it explicit only if the keyword value is
// already an object. Otherwise leave the boolean form alone.
if let Some(Value::Object(_)) = self.object.get(&k) {
// The keyword value is an object; set encoding inside it.
// For the POC we don't reach this branch (boolean form only).
// Only emit `encoding` when the keyword is already in
// object form — the boolean shorthand already means
// length-prefixed (the default), so emitting it would
// be redundant noise.
if let Some(Value::Object(obj)) = self.object.get_mut(&k) {
obj.insert(
"encoding".to_string(),
Value::String("length-prefixed".to_string()),
);
}
}
VariableEncoding::OffsetIndirect => {
// Replace the boolean-true form with the object form
// carrying the encoding annotation.
self.object.insert(k, Value::Object(Map::from_iter([
("encoding".to_string(), Value::String("offset-indirect".to_string())),
])));
// carrying the encoding annotation. If the keyword is
// already an object, update its `encoding` entry.
match self.object.get_mut(&k) {
Some(Value::Object(obj)) => {
obj.insert(
"encoding".to_string(),
Value::String("offset-indirect".to_string()),
);
}
_ => {
self.object.insert(
k,
Value::Object(Map::from_iter([
(
"encoding".to_string(),
Value::String("offset-indirect".to_string()),
),
])),
);
}
}
}
}
}
@@ -640,6 +666,42 @@ mod tests {
);
}
#[test]
fn builder_encoding_length_prefixed_on_boolean_form_is_noop() {
// The boolean-true shorthand already means length-prefixed (the
// default), so `.encoding(LengthPrefixed)` on a fresh constructor
// is a no-op — no `encoding` key is emitted.
let s = Schema::string().encoding(VariableEncoding::LengthPrefixed).build();
assert_eq!(s, json!({"AlkType:String": true}));
}
#[test]
fn builder_encoding_length_prefixed_overwrites_offset_indirect() {
// Switching from offset-indirect back to length-prefixed updates
// the `encoding` entry inside the keyword's object form in place.
let s = Schema::string()
.encoding(VariableEncoding::OffsetIndirect)
.encoding(VariableEncoding::LengthPrefixed)
.build();
assert_eq!(
s,
json!({"AlkType:String": {"encoding": "length-prefixed"}})
);
}
#[test]
fn builder_encoding_offset_indirect_overwrites_length_prefixed() {
// And the reverse: length-prefixed (object form) -> offset-indirect.
let s = Schema::string()
.encoding(VariableEncoding::LengthPrefixed)
.encoding(VariableEncoding::OffsetIndirect)
.build();
assert_eq!(
s,
json!({"AlkType:String": {"encoding": "offset-indirect"}})
);
}
#[test]
fn builder_align_sets_annotation() {
let s = Schema::struct_().align(256).build();
@@ -660,7 +722,8 @@ mod tests {
#[test]
fn builder_chunk_header_compiles_in_packed_mode() {
// The POC's primary use case: channels' 8-byte chunk header.
// channels' 8-byte chunk header: the primary binary-layout use
// case for the builder. Big-endian, packed mode.
let mut schema = Schema::struct_()
.endian(Endian::Big)
.field("channel_id", Schema::uint32())