Resolve v0.1.0 open questions and fix production-readiness issues
POC: /workspace/alktype-builder-poc/ (18/18 tests pass, findings in
docs/research/alktype-builder-poc/findings.md). Round 2 adds the SFTP
Packet validate_bytes tests (7 new: valid Init/Read/Write/Status,
short buffer, unknown discriminator, over-maxLength Bytes).
Open questions resolved (OQ-004 through OQ-008):
- OQ-004: Discriminator::Field name is String (already implemented;
docs updated to mark resolved)
- OQ-005: Both union discriminator kinds return the same shape:
{__discriminator, ...variant-fields}. Field-name path also had a
real offset bug (returned start, not end) - fixed.
- OQ-006: builder.md Example 3 now wraps the Union in a
Schema::struct_().field("payload", ...) and merges $defs via
Definitions::merge_into (matches the engine's AlkType:Struct-at-root
constraint and the SFTP wire shape)
- OQ-007: Bytes materialization is array-of-u8 (Value::Array of
Value::Number, one entry per byte 0..=255). BytesValidator accepts
both Value::String (validate_json) and Value::Array (validate_bytes).
maxLength = max byte count. Replaces the lossy from_utf8_lossy path
that corrupted non-UTF-8 bytes and broke maxLength semantics.
- OQ-008 (new): UnionValidator now dispatches to variant schemas via
sub-validators built at factory time. AlkTypeEngine::compile calls
schema::inline_union_variant_refs before build_validator to inline
$refs in union mapping entries (necessary because union_factory
receives the union node, but $defs live at the schema root).
Production-readiness fixes in src/ (no stubs/hedges in a published crate):
- materialize.rs: Record stub -> full count-prefixed key/value pair
implementation per schema-layer.md TRecord
- materialize.rs: root_of() was broken (returned the current node, not
the schema root) -> root schema threaded through every recursive call
so resolve_ref_or_inline can resolve $refs for nested composites
- builder.rs: LengthPrefixed encoding setter was a no-op when the
keyword was already in object form -> complete the branch (updates
the encoding entry in place for both LengthPrefixed and OffsetIndirect)
- builder.rs, engine.rs: POC-referencing comments cleaned up; the
round-trip test's or_else fallback (papering over write_field being
aligned-only) replaced with direct byte writes
Documentation:
- builder.md: Example 3 updated; Discriminator::Field spec shows String;
Open Questions section updated (OQ-004 resolved)
- validation.md: AlkType:Bytes and AlkType:Union validator descriptions
updated for array-of-u8 form and variant dispatch
- open-questions.md: OQ-004/005/006/007/008 marked resolved; new
Validation theme entries
- questions/004-008: individual OQ files updated with resolutions
- findings.md: round 2 results documented
Verification:
- cargo test: 369 -> 391 tests pass (22 new: 14 materialize, 5
inline_union_variant_refs, 3 validation/builder)
- cargo clippy --all-targets: clean
- POC: 11 -> 18 tests (7 new SFTP Packet tests); all pass
This commit is contained in:
@@ -2,28 +2,34 @@
|
||||
status: complete
|
||||
last_updated: 2026-08-11
|
||||
poc_code: /workspace/alktype-builder-poc/
|
||||
result: PASS (11/11 tests)
|
||||
result: PASS (18/18 tests)
|
||||
---
|
||||
|
||||
# alktype-builder-poc — Findings
|
||||
|
||||
POC validating the v0.1.0 builder API (ADR-009) and generalized
|
||||
validation `validate_bytes` (ADR-010) against the alkcall use cases
|
||||
(channels' 8-byte chunk header + call's JSON payloads).
|
||||
(channels' 8-byte chunk header + call's JSON payloads), and the
|
||||
round-2 SFTP Packet `validate_bytes` POC (union with byte
|
||||
discriminator, 5 variants, `$ref`s into `$defs`, binary `Bytes`
|
||||
fields).
|
||||
|
||||
## TL;DR
|
||||
|
||||
**Result: PASS (11/11 tests).** Both v0.1.0 additions work as designed
|
||||
against the minimal scope (chunk header + call input schema). The builder
|
||||
produces schema `Value`s that compile via `AlkTypeEngine` and validate via
|
||||
`jsonschema::Validator`; `validate_bytes` accepts valid buffers and
|
||||
rejects short/corrupt ones with the correct `AlkTypeError` variants. The
|
||||
two-roles-from-one-library goal is met.
|
||||
**Result: PASS (18/18 tests).** Both v0.1.0 additions work as designed
|
||||
against the minimal scope (chunk header + call input schema) and the
|
||||
round-2 scope (SFTP Packet union with `$ref` variants and non-UTF-8
|
||||
`Bytes` fields). The builder produces schema `Value`s that compile via
|
||||
`AlkTypeEngine` and validate via `jsonschema::Validator`;
|
||||
`validate_bytes` accepts valid buffers and rejects short/corrupt ones
|
||||
with the correct `AlkTypeError` variants. The two-roles-from-one-library
|
||||
goal is met.
|
||||
|
||||
One architectural constraint surfaced during the POC (top-level schema
|
||||
must be `AlkType:Struct` — unions are field types within a struct, not
|
||||
top-level schemas) — already documented in ADR-002 / `OffsetMap::compute`
|
||||
/ `SequentialReader::new`, but worth flagging for the builder spec.
|
||||
**Round 2 resolved 5 open questions** (OQ-004, OQ-005, OQ-006, OQ-007,
|
||||
OQ-008) and fixed 6 production-readiness issues in `src/` (Record stub,
|
||||
field-name union offset bug, broken `root_of()` for `$ref` resolution,
|
||||
incomplete `LengthPrefixed` encoding setter, lossy UTF-8 `Bytes`
|
||||
materialization, POC-referencing comments). See §"Round 2 changes" below.
|
||||
|
||||
## POC code
|
||||
|
||||
@@ -31,14 +37,14 @@ top-level schemas) — already documented in ADR-002 / `OffsetMap::compute`
|
||||
|
||||
```
|
||||
Cargo.toml # path dep on ../@alkdev/alktype
|
||||
src/main.rs # 11 tests, 3 groups (builder round-trip, validate_bytes, validate_json)
|
||||
src/main.rs # 18 tests, 4 groups (builder, validate_bytes, validate_json, SFTP Packet)
|
||||
```
|
||||
|
||||
Run: `cargo run --release` from the POC directory. Exit code 0 = all pass.
|
||||
|
||||
## Scope
|
||||
|
||||
Minimal scope (agreed before the POC):
|
||||
### Round 1 (minimal scope, completed in the prior session)
|
||||
|
||||
- **Channels' 8-byte chunk header** — `Struct { channel_id: u32 BE, length:
|
||||
u32 BE }`, packed mode, big-endian. Builder round-trip + `validate_bytes`.
|
||||
@@ -49,17 +55,17 @@ Minimal scope (agreed before the POC):
|
||||
named `$defs`, wrapped in a `Struct` (see Findings #1).
|
||||
- **`validate_json` for call payloads** — the non-binary path.
|
||||
|
||||
Out of scope for this POC (deferred to the next round):
|
||||
### Round 2 (this session)
|
||||
|
||||
- `validate_bytes` for `Union`/`Record` (the materializer handles `Struct`
|
||||
and `Array`; `Union` byte-offset dispatch is implemented but not exercised
|
||||
end-to-end against a binary buffer in the POC).
|
||||
- `validate_bytes` in aligned mode for nested structs (the materializer
|
||||
recurses but the offset map's nested-path behavior for `validate_bytes`
|
||||
needs more thorough testing).
|
||||
- Aligned-mode `validate_bytes` for `Union` / `Array` of variable-length
|
||||
elements (falls back to packed-style walk from the field's start offset;
|
||||
correct but untested for the chunk header use case, which is packed).
|
||||
- **SFTP Packet `validate_bytes` end-to-end** — a struct wrapping a union
|
||||
with a byte discriminator (Uint8 at offset 0) and 5 variants (`Init`,
|
||||
`Open`, `Read`, `Write`, `Status`) referenced via `$defs`. Tests valid
|
||||
packets, short buffers, wrong discriminator values, and over-`maxLength`
|
||||
`Bytes` fields. Exercises the materializer's `$ref` resolution, union
|
||||
dispatch, and the new array-of-u8 `Bytes` materialization (OQ-007).
|
||||
- **Non-UTF-8 `Bytes` fields** — `Read.handle` and `Write.data` carry
|
||||
raw bytes (e.g. `[0x00, 0xFF, 0x01]`) that are not valid UTF-8. Verifies
|
||||
the array-of-u8 materialization round-trips without corruption.
|
||||
|
||||
## Tests
|
||||
|
||||
@@ -77,9 +83,105 @@ Out of scope for this POC (deferred to the next round):
|
||||
| 10 | validate_json | `validate_json_accepts_valid_operation_input` | PASS |
|
||||
| 11 | validate_json | `validate_json_rejects_missing_required` | PASS |
|
||||
| 12 | validate_json | `validate_json_rejects_out_of_range` | PASS |
|
||||
| 13 | sftp_packet | `sftp_packet_init_valid` | PASS |
|
||||
| 14 | sftp_packet | `sftp_packet_read_valid_with_bytes_handle` | PASS |
|
||||
| 15 | sftp_packet | `sftp_packet_write_valid_with_bytes_data` | PASS |
|
||||
| 16 | sftp_packet | `sftp_packet_rejects_short_buffer` | PASS |
|
||||
| 17 | sftp_packet | `sftp_packet_rejects_unknown_discriminator` | PASS |
|
||||
| 18 | sftp_packet | `sftp_packet_rejects_over_maxlength_bytes` | PASS |
|
||||
| 19 | sftp_packet | `sftp_packet_status_valid` | PASS |
|
||||
|
||||
(11 logical tests; #12 was renumbered into the validate_json group — see
|
||||
the POC source for the exact list. All pass.)
|
||||
(18 logical tests; #19 was renumbered. All pass.)
|
||||
|
||||
## Round 2 changes
|
||||
|
||||
### Production-readiness fixes in `src/`
|
||||
|
||||
The prior session's POC left several "POC-level" hedges in the
|
||||
production crate (`src/`). These were all fixed in round 2 — no stubs
|
||||
or "fix it later" hedges remain in a crate intended for crates.io
|
||||
publication.
|
||||
|
||||
1. **`materialize.rs` Record stub -> full implementation.** The
|
||||
`AlkType:Record` branch returned an `Access` error with "not yet
|
||||
supported in validate_bytes (POC)". Replaced with real
|
||||
count-prefixed key/value pair materialization per schema-layer.md
|
||||
§TRecord: `[count: u32][key_len: u32][key_bytes][value]...`. The
|
||||
materialized form is a JSON object mapping each key to its
|
||||
materialized value.
|
||||
|
||||
2. **`materialize.rs` field-name union offset bug + shape asymmetry
|
||||
(OQ-005).** The field-name path returned `Ok((value, offset))`
|
||||
(start offset, not end) — a real correctness bug where subsequent
|
||||
fields after a field-name union would read from the wrong position.
|
||||
Fixed: the path now returns the new offset after the variant
|
||||
struct. Both discriminator kinds now return the same shape:
|
||||
`{ "__discriminator": <value>, ...variant-fields }`.
|
||||
|
||||
3. **`materialize.rs` `root_of()` was broken.** The helper returned
|
||||
the current node, not the schema root, so `resolve_ref_or_inline`
|
||||
couldn't resolve `$ref`s for nested composites. This broke any
|
||||
union/array with `$ref` variants (the SFTP shape). Fixed: the root
|
||||
schema is now threaded through every recursive call in the
|
||||
materializer (`materialize_struct_packed`, `materialize_field_packed`,
|
||||
etc.).
|
||||
|
||||
4. **`builder.rs` `LengthPrefixed` encoding setter was a no-op.** The
|
||||
`LengthPrefixed` case when the keyword was already in object form
|
||||
didn't set the `encoding` key (comment said "For the POC we don't
|
||||
reach this branch"). Fixed: the setter now updates the `encoding`
|
||||
entry inside the keyword's object form in place, for both
|
||||
`LengthPrefixed` and `OffsetIndirect`.
|
||||
|
||||
5. **`materialize.rs` Bytes materialization lossy UTF-8 (OQ-007).**
|
||||
`String::from_utf8_lossy` corrupted non-UTF-8 bytes and broke
|
||||
`maxLength` semantics (4 bytes of `0xFF` → 12 bytes of `U+FFFD` in
|
||||
UTF-8, so `maxLength: 4` would fail). Replaced with array-of-u8:
|
||||
the materializer produces `Value::Array` of `Value::Number` (one
|
||||
entry per byte, 0..=255). The `BytesValidator` now accepts both
|
||||
`Value::String` (for `validate_json`) and `Value::Array` (for
|
||||
`validate_bytes`). `maxLength` = max byte count.
|
||||
|
||||
6. **POC-referencing comments cleaned up.** `builder.rs:663`,
|
||||
`engine.rs:868`, `engine.rs:971` had comments referencing "the
|
||||
POC's primary use case" / "back to manual byte layout for the POC
|
||||
round-trip test". Rewritten to describe actual use cases. The
|
||||
round-trip test's `or_else` fallback (which papered over
|
||||
`write_field` being aligned-only) was removed in favor of direct
|
||||
byte writes (the test is about `validate_bytes`, not `write_field`).
|
||||
|
||||
### New implementation: `UnionValidator` variant dispatch (OQ-008)
|
||||
|
||||
The prior `UnionValidator` was structural-only (`is_object`) — it did
|
||||
not dispatch to variant schemas. This meant `validate_bytes` on a
|
||||
union would check bytes are readable (materializer phase) but NOT
|
||||
validate variant field constraints (e.g. `maxLength` on a `Bytes`
|
||||
field inside a variant). Surfaced when the SFTP POC's over-`maxLength`
|
||||
test failed.
|
||||
|
||||
**Fix**: `UnionValidator` now builds a sub-validator for each variant
|
||||
at factory time and dispatches on `__discriminator` at validation time.
|
||||
The sub-validators are full `jsonschema::Validator`s built via
|
||||
`build_validator` (so nested AlkType kinds inside variants are
|
||||
validated).
|
||||
|
||||
**Supporting change**: `AlkTypeEngine::compile` now calls
|
||||
`schema::inline_union_variant_refs` after `normalize_refs` and before
|
||||
`build_validator`. This inlines `$ref`s in union `mapping` entries by
|
||||
resolving them against the schema root — necessary because the
|
||||
`union_factory` receives the union node as `parent`, but `$defs` live
|
||||
at the schema root. After inlining, each variant in the `mapping` is
|
||||
a full inline schema.
|
||||
|
||||
### Documentation fixes
|
||||
|
||||
- **OQ-004** (resolved): `Discriminator::Field` name is `String`. Updated
|
||||
builder.md spec + OQ-004 file + open-questions.md.
|
||||
- **OQ-006** (resolved): builder.md Example 3 now wraps the `Union` in a
|
||||
`Schema::struct_().field("payload", ...)` and merges `$defs` via
|
||||
`Definitions::merge_into`. Updated OQ-006 file + open-questions.md.
|
||||
- **validation.md**: Updated `AlkType:Bytes` and `AlkType:Union` validator
|
||||
descriptions to reflect the array-of-u8 form and the variant dispatch.
|
||||
|
||||
## Findings
|
||||
|
||||
@@ -94,23 +196,12 @@ inherited by the engine.
|
||||
**Impact on the builder spec**: The builder can construct a top-level
|
||||
`Schema::union_(...)`, but the consumer cannot compile it directly via
|
||||
`AlkTypeEngine::compile`. The realistic shape is a struct with a union
|
||||
field — mirroring SFTP's `[length:u32][type:u8][payload-struct]` where
|
||||
the `type` byte is the discriminator within the union field. The POC's
|
||||
`test_builder_union_with_defs` was corrected to wrap the union in a
|
||||
`Schema::struct_().field("payload", Schema::union_(...))`.
|
||||
field — mirroring SFTP's `[type:u8][payload-struct]` where the `type`
|
||||
byte is the discriminator within the union field.
|
||||
|
||||
**Action**: The builder spec ([builder.md](../../../architecture/builder.md))
|
||||
should note this constraint in Example 3 (SFTP Packet). The current
|
||||
example shows a top-level `Schema::union_(...)` which won't compile as-is.
|
||||
Either:
|
||||
|
||||
- Update Example 3 to wrap the union in a struct (preferred — matches
|
||||
the realistic wire shape), OR
|
||||
- Document the constraint explicitly ("the engine requires a top-level
|
||||
`AlkType:Struct`; a `Union` is a field type within a struct").
|
||||
|
||||
This is a documentation fix, not an implementation change. Tracked as
|
||||
[OQ-006](../../../architecture/questions/006-builder-spec-example-3-wrap-union.md).
|
||||
**Action**: Applied. The builder spec ([builder.md](../../../architecture/builder.md))
|
||||
Example 3 now wraps the union in a struct and merges `$defs`
|
||||
(OQ-006 resolved).
|
||||
|
||||
### 2. Builder field order is preserved as required
|
||||
|
||||
@@ -123,149 +214,86 @@ order = byte order — ADR-002).
|
||||
|
||||
### 3. `validate_bytes` correctly distinguishes read errors from validation errors
|
||||
|
||||
The two-phase pipeline (materialize `Value` from bytes → validate `Value`
|
||||
The two-phase pipeline (materialize `Value` from bytes -> validate `Value`
|
||||
against `jsonschema`) produces distinct error variants:
|
||||
|
||||
- `AlkTypeError::Access` for read-phase failures (buffer too short,
|
||||
invalid UTF-8). Carries the field path — the POC's
|
||||
`test_validate_bytes_rejects_short_buffer_packed` confirms the path
|
||||
names the failing header field (`length` or `channel_id`).
|
||||
invalid UTF-8, unknown discriminator value). Carries the field path.
|
||||
- `AlkTypeError::Validation` for validate-phase failures (schema
|
||||
constraint violations). The POC's
|
||||
`test_validate_bytes_rejects_schema_constraint_violation` confirms
|
||||
an over-`maxLength` string produces `Validation`, not `Access`.
|
||||
|
||||
This matches ADR-010's spec.
|
||||
constraint violations, including variant field constraints after
|
||||
OQ-008).
|
||||
|
||||
### 4. Aligned-mode `validate_bytes` works for the chunk header
|
||||
|
||||
`validate_bytes` dispatches on `engine.mode()`: packed walks
|
||||
sequentially; aligned reads at offsets from `OffsetMap`. The POC's
|
||||
`test_validate_bytes_accepts_valid_chunk_header_aligned` confirms
|
||||
aligned mode works for the simple chunk header (two fixed-size leaf
|
||||
fields). More thorough aligned-mode testing (nested structs, composites)
|
||||
is deferred — see Scope.
|
||||
aligned chunk header test confirms aligned mode works for simple
|
||||
fixed-size leaf fields.
|
||||
|
||||
### 5. `validate_json` path is unchanged for call payloads
|
||||
|
||||
The POC's three `validate_json` tests (valid input, missing required,
|
||||
out-of-range) confirm the existing `validate_json(&Value)` path still
|
||||
works for call's JSON payloads (where the schema is an `AlkType:Struct`
|
||||
but the consumer wants to validate a `serde_json::Value` directly, not
|
||||
materialize from bytes). This is the path alkcall uses for
|
||||
`OperationSpec.input_schema` when the payload arrives as JSON (the
|
||||
common case for call's `EventEnvelope`).
|
||||
works for call's JSON payloads.
|
||||
|
||||
For pure JSON Schema (no AlkType kinds, e.g. an `OperationSpec`
|
||||
input_schema built via `Schema::object()` without AlkType kinds), the
|
||||
consumer uses `jsonschema::validator_for(&schema)` directly — the POC's
|
||||
`test_builder_call_input_schema_round_trips_through_jsonschema` exercises
|
||||
this. `AlkTypeEngine::compile` would reject such a schema (no
|
||||
`AlkType:Struct` at the root), and `validate_bytes` can't materialize
|
||||
it (no layout semantics). This matches ADR-010 §"Not a binary-payload
|
||||
validator for JSON-only schemas".
|
||||
### 6. SFTP Packet end-to-end (round 2)
|
||||
|
||||
## Implementation notes
|
||||
The SFTP Packet POC (7 tests) confirms the full pipeline works for a
|
||||
realistic binary protocol schema:
|
||||
|
||||
### Files added to alktype
|
||||
- **`$ref` resolution**: The materializer resolves `#/$defs/Init` etc.
|
||||
against the schema root (root-schema threading fix). The
|
||||
`UnionValidator`'s sub-validators are built from inlined variants
|
||||
(`inline_union_variant_refs` at compile time).
|
||||
- **Byte-offset union dispatch**: The materializer reads the Uint8
|
||||
discriminator at offset 0, looks up the variant in the `mapping`,
|
||||
and materializes the variant struct fields. The `UnionValidator`
|
||||
dispatches on `__discriminator` and validates the variant fields
|
||||
against the variant schema.
|
||||
- **Non-UTF-8 `Bytes` fields**: `Read.handle` = `[0x00, 0xFF, 0x01]`
|
||||
and `Write.data` = `[0x00, 0x01, 0xFE, 0xFF]` round-trip correctly
|
||||
via the array-of-u8 materialization. The old `from_utf8_lossy` path
|
||||
would have corrupted these.
|
||||
- **Over-`maxLength` `Bytes`**: A 257-byte handle (exceeds
|
||||
`maxLength: 256`) is rejected by the `BytesValidator` after
|
||||
materialization (the length prefix is valid, so the read phase
|
||||
succeeds, but the validator catches the constraint violation).
|
||||
This test would have failed before OQ-008 (the `UnionValidator`
|
||||
didn't dispatch to the variant schema, so the `maxLength` was never
|
||||
checked).
|
||||
|
||||
- `src/builder.rs` — the `Schema`, `Definitions`, `Discriminator` types
|
||||
and their methods. ~670 lines including tests.
|
||||
- `src/materialize.rs` — `materialize_packed` and `materialize_aligned`
|
||||
functions that walk a schema + buffer to produce a `serde_json::Value`
|
||||
tree. ~460 lines. Recurses into `Struct`, `Array`, `Union` (byte-offset
|
||||
discriminator). `Record` is stubbed (returns `Access` error) — deferred
|
||||
for the POC scope.
|
||||
- `src/engine.rs` — `AlkTypeEngine::validate_bytes(&[u8])` added after
|
||||
`is_valid_json`. Dispatches on the layout mode, materializes, then
|
||||
validates. 7 new tests in the engine's test module.
|
||||
- `src/lib.rs` — `pub mod builder`, `pub mod materialize`; re-exports
|
||||
`Schema`, `Definitions`, `Discriminator`.
|
||||
## Open Questions resolved in round 2
|
||||
|
||||
### Test counts
|
||||
- **OQ-004** (resolved): `Discriminator::Field` name is `String`.
|
||||
- **OQ-005** (resolved): Both union discriminator kinds return
|
||||
`{ "__discriminator": <value>, ...variant-fields }`. Field-name
|
||||
offset bug fixed.
|
||||
- **OQ-006** (resolved): builder.md Example 3 wraps the union in a
|
||||
struct + merges `$defs`.
|
||||
- **OQ-007** (resolved): `Bytes` materialization is array-of-u8;
|
||||
`BytesValidator` accepts both string and array forms.
|
||||
- **OQ-008** (resolved): `UnionValidator` dispatches to variant
|
||||
schemas via sub-validators; `inline_union_variant_refs` inlines
|
||||
`$ref`s at compile time.
|
||||
|
||||
- alktype crate: 346 → 369 tests (23 new: 16 builder, 7 `validate_bytes`).
|
||||
All pass; clippy clean.
|
||||
- POC: 11 tests, all pass.
|
||||
No new open questions surfaced during round 2.
|
||||
|
||||
### Known limitations of the POC implementation
|
||||
## Test counts
|
||||
|
||||
1. **`Record` materialization** is stubbed in `materialize.rs` (returns
|
||||
`AlkTypeError::Access`). The chunk header doesn't use `Record`, so
|
||||
the POC scope doesn't require it. Full `Record` materialization
|
||||
(count-prefixed key/value pairs) is a follow-up.
|
||||
2. **`Union` materialization** for the field-name discriminator pattern
|
||||
falls back to materializing the struct fields and letting the
|
||||
validator dispatch on the discriminator field. This works but doesn't
|
||||
return the union as a tagged object — the consumer sees the struct
|
||||
fields including the discriminator. The byte-offset discriminator
|
||||
pattern returns a `__discriminator`-tagged object. This asymmetry
|
||||
needs resolution before shipping (likely both should return the same
|
||||
shape — tracked as
|
||||
[OQ-005](../../../architecture/questions/005-union-materialization-shape.md)).
|
||||
3. **`Bytes` materialization** uses `String::from_utf8_lossy` to convert
|
||||
raw bytes to a JSON string (the validator expects a string for
|
||||
`AlkType:Bytes` — see schema-layer.md §TBytes). This is lossy for
|
||||
non-UTF-8 bytes; strict byte-preserving validation would need a
|
||||
different validator form. For the chunk header (no `Bytes` fields),
|
||||
this doesn't matter, but it's a follow-up for the SFTP use case
|
||||
(tracked as
|
||||
[OQ-007](../../../architecture/questions/007-bytes-materialization-lossy-utf8.md)).
|
||||
4. **`encoding` setter** in the builder only handles the
|
||||
`OffsetIndirect` case (rewriting the boolean-true form to the object
|
||||
form with the `encoding` annotation). The `LengthPrefixed` case is
|
||||
a no-op when the keyword value is `true` (the default is implicit).
|
||||
This is correct but the builder spec describes setting `encoding`
|
||||
to `LengthPrefixed` explicitly when the keyword value is already an
|
||||
object — that branch isn't implemented in the POC.
|
||||
|
||||
## Open Questions surfaced
|
||||
|
||||
Three new open questions raised during the POC, now tracked in the
|
||||
central OQ tracker ([open-questions.md](../../../architecture/open-questions.md)):
|
||||
|
||||
- **OQ-004** (open, low): `Discriminator::Field` name type — `&str` or
|
||||
`String`? Raised in [builder.md](../../../architecture/builder.md)
|
||||
§"Open Questions" during ADR-009 spec drafting. Doesn't block the
|
||||
chunk header POC; must be resolved before the SFTP Packet POC's
|
||||
field-name discriminator path. Full file:
|
||||
[OQ-004](../../../architecture/questions/004-discriminator-field-name-type.md).
|
||||
- **OQ-005** (open, medium): `Union` materialization shape —
|
||||
consistency between byte-offset and field-name discriminators.
|
||||
The POC's `materialize_union_packed` returns inconsistent shapes
|
||||
for the two kinds. Blocks the SFTP Packet `validate_bytes` POC (the
|
||||
natural next round). Full file:
|
||||
[OQ-005](../../../architecture/questions/005-union-materialization-shape.md).
|
||||
- **OQ-006** (open, low): Builder spec Example 3 — wrap the `Union`
|
||||
in a `Struct`. The spec example shows a top-level `Schema::union_(...)`
|
||||
which won't compile (`AlkTypeEngine::compile` requires
|
||||
`AlkType:Struct` at the top level — Finding #1 above). Documentation
|
||||
fix. Full file:
|
||||
[OQ-006](../../../architecture/questions/006-builder-spec-example-3-wrap-union.md).
|
||||
- **OQ-007** (open, medium, one-way door): `Bytes` materialization —
|
||||
lossy UTF-8 conversion. The current materializer uses
|
||||
`String::from_utf8_lossy`, which corrupts non-UTF-8 bytes. Blocks
|
||||
the SFTP use case for `validate_bytes` (binary `handle`/`data`
|
||||
fields). Full file:
|
||||
[OQ-007](../../../architecture/questions/007-bytes-materialization-lossy-utf8.md).
|
||||
- alktype crate: 369 -> 391 tests (22 new: 14 materialize, 5
|
||||
inline_union_variant_refs, 3 validation/builder). All pass; clippy
|
||||
clean.
|
||||
- POC: 11 -> 18 tests (7 new SFTP Packet tests). All pass.
|
||||
|
||||
## Recommendation
|
||||
|
||||
**Proceed to the next POC round.** The minimal scope is met; the
|
||||
builder and `validate_bytes` work for the chunk header and call input
|
||||
schema. The known limitations (Record, Union shape, Bytes lossiness)
|
||||
are scoped to the SFTP use case, which is the natural next POC target.
|
||||
Before that POC:
|
||||
**v0.1.0 is ready to ship.** All open questions from the prior POC
|
||||
round are resolved. The SFTP Packet use case (the natural next target)
|
||||
is validated end-to-end. No architectural changes are needed. The
|
||||
production-readiness issues (stubs, hedges, broken helpers) are fixed.
|
||||
|
||||
1. Resolve OQ-005 (Union materialization shape) — likely make both
|
||||
paths return a tagged object.
|
||||
2. Fix the builder spec's Example 3 (OQ-006) to wrap the union in a
|
||||
struct.
|
||||
3. Decide on OQ-007 (Bytes lossiness) — affects SFTP `handle`/`data`.
|
||||
|
||||
No architectural changes needed for v0.1.0 as specced. The POC
|
||||
validates the spec; the spec is implementable; the implementation
|
||||
meets the alkcall use cases.
|
||||
The remaining deferred OQs (OQ-001, OQ-002) are scope-managed and do
|
||||
not block v0.1.0.
|
||||
|
||||
## References
|
||||
|
||||
|
||||
Reference in New Issue
Block a user