The bast-pivot.md research doc had grown to 1477 lines (~58KB) through
iterative editing, pushing its most actionable content (D-BAST
decisions, POC result, migration steps) past the 50KB Read tool cap.
Agents peeking at the truncated file landed in duplicated/out-of-order
sections. Decompose into three readable-sized files with distinct roles:
- docs/architecture/bast-format.md (28KB, new): the normative BAST
format spec -- meta-schema, TypeRef, examples, validation model.
Grounded in the POC and D-BAST-001..009. Stable and safe to write
now; schema-layer.md/validation.md stay describing current code and
are rewritten post-implementation (per AGENTS.md ADR-grounding rule).
- docs/plans/bast-implementation.md (31KB, new): the execution entry
point -- ordered 10-step plan with per-step goal/files/spec-ref/
verification, the public-API semver contract table up front as a
scope-creep guardrail, and the ADR-sync checklist at the end. Each
step links to the specific bast-format.md section and D-BAST anchor.
- docs/research/bast-pivot.md (28KB, trimmed): now the research record
only -- Summary, Motivation, POC scope/result, Decisions, Risks,
References. The normative format spec, what-changes tables,
validator-split details, and migration steps moved to the two new
docs; pointers added. 1155 lines removed, 216 added.
- docs/architecture/README.md: index updated to list bast-format.md
and the two in-progress pivot docs, with notes on schema-layer.md
and validation.md being rewritten when the pivot lands.
All three files are under the 50KB Read cap, so an implementing agent
gets the whole document in one call. Cross-reference anchors verified
to resolve. No code changes; cargo test --release (396 tests) green.
Verification: cargo test --release (310 crate + 86 integration, all pass).
Scope-creep guardrail for the public API during implementation. Maps
every item re-exported from src/lib.rs to a class (breaking / additive /
unchanged) with the specific change, and every ADR (001-010) to an
action (supersede / amend / unchanged) with the reason.
Net breaking: compile (signature), validate_json/is_valid_json
(contract), Schema::build/Definitions::build (output format),
build_validator (signature or removal), and the ~13 schema::* helper
re-exports. Net additive: BAST parser, BAST-native validator,
AlkTypeKind::from_str/to_str. Net unchanged: the entire layout +
data-access + materialize + tunion layer, AlkTypeError (D-BAST-009),
the Discriminator builder, AlkTypeKind variants.
Flags three small decisions deferred to their implementation steps
(validate_json JSON Schema source, build_validator fate, schema::*
re-export retention) so they don't become drive-by semver changes.
This is a living guide — it may shift slightly during implementation,
but capturing the contract now prevents public-surface drift. Doc-only.
Converts the open question into a closed decision. Rationale: consumer
ergonomics on the combined validate_json + validate_bytes path — one
uniform payload type means one match arm downstream. Option 2
(Validation(String)) would force validate_json to flatten its
structured errors to a String, losing information on the richer path to
accommodate the less rich one. The no_std/minimal-build angle that
option 2 was meant to enable is moot: validate_json requires jsonschema
regardless, so a bytes-only no_std build already has to give up
validate_json as a separate larger decision; dropping the type from one
error variant doesn't unlock it.
Updates Phase 1 step 5 to reference D-BAST-009 for the error
construction pattern, and rewrites POC Result observation 4 from
'deferred decision' to 'decided — see D-BAST-009'.
No semver-relevant change to the Validation variant. Doc-only.
Adds the POC Result section (POC on branch bast-validator-poc, commit
f371fe4 — 20/20 tests, full 416-test suite green, clippy/wasm/doc clean).
Hypothesis confirmed: a recursive walker over the BAST type tree fully
replaces the 19 custom keyword validators on the validate_bytes path,
recovers OQ-008 union variant dispatch, and fixes the enum-membership
dead constraint. The POC code is reference scaffolding on the branch
and is not merged to main — it is superseded by Phase 1 step 5.
Marks Phase 1 step 2 and the POC Scope section as done with pointers to
the result section.
Elevates the deferred AlkTypeError::Validation payload-shape question
to OQ-BAST-001: keep jsonschema::ValidationError<'static> (POC choice,
simplest, dependency stays) vs introduce Validation(String) (drops
jsonschema from the error type; semver-relevant public-API change).
Decision belongs to the production refactor.
Verification: doc-only change, no code touched.
- Rewrite Validator Split around BAST-native validator for
validate_bytes (walks BAST, checks value-domain constraints, no
external JSON Schema needed); validate_json uses standard
jsonschema::Validator from consumer-provided JSON Schema
- Add targeted POC: BAST-native validator replacing 19 custom keyword
validators on the bytes path, verified via existing test suite
- Fix meta-schema: require count on arrays (variable-element arrays
deferred per OQ-001), add optional fields array to UnionDef for
field-name discriminators
- Remove lying no-count array example, replace with deferred note
- Document dead enum constraint (materialized index never matches
string-membered enum); BAST-native validator fixes it via index
bounds check
- Resolve all 7 OQs + 3 spec gaps as D-BAST-001 through D-BAST-008
- Update Migration Path, Risks table, Engine internals to reflect
BAST-native validator
- Clarify 'no pocs needed' was an overcorrection: layout swap needs
no POC, but the validation model does
Verification: docs-only change, no code affected
- Remove the four proposed POCs: they were implementation smoke tests,
not de-risking probes. The pivot is a backend swap on a proven layout
engine; byte-identity is already proven and the layout code is
unchanged, so there is nothing empirical left to de-risk.
- Remove the recursive TreeNode example and the recursion mention in
design principle 2: recursion is not a binary-layout concern and the
engine has no cycle detection.
- Add a Spec Gaps section: validate_bytes semantics after keyword
validator removal (UnionValidator variant dispatch regression),
arrays of variable-length elements (engine rejects them), and
field-name discriminator unions (meta-schema cannot express them).
- Reframe the engine change as an accessor-layer refactor: the
walkers' (kind, field list, annotations) reads change; everything
beneath them carries over unchanged.
Correct the validator split section to clarify that jsonschema is
not removed — it remains the JSON Schema validator for both paths
(BAST meta-schema validation and standard JSON payload validation).
Only the custom keyword registration path is removed. Also fix the
build_validator and validate_json entries in the engine changes
table to reflect that they are repurposed, not removed.
Proposes replacing custom JSON Schema keywords with a standalone
kind-based JSON format (BAST) using / for composition.
Covers format design, meta-schema, engine changes, validator split,
codegen future, ABI adapter potential, migration path, 7 open
questions, and 4 proposed POCs.
glm-5.2
deepseek-v4-pro