- overview.md, core.md, emit.md, targets.md: modular spec set for the generalized engine (contexts + templates + partials -> text), with the three targets (ssr-html, bast-rust-codecs, call-handlers, wgsl-jit) as template sets plus emit postures - ADRs 001-010: one core/template-set split, handlebars 6.4 strict substrate, sync/wasm/deterministic posture, typed contexts with builders, per-registry escaper, SSR library posture, typed direct codecs output contract, template-set ownership, feature gating, library-only v1 - open-questions.md: OQ-01..OQ-08 centralized tracker carrying phase-0 residue (GN-01/02/07/09) with three deferred(scope) entries plus a resolved disposition for phase-0 OQ-GN-04 - tasks/architecture/: external-trigger tracker tasks paired with the deferred OQs (taskgraph validate passes) - phase-0.md checklist dispositioned for OQ-GN-04; AGENTS.md lifecycle updated to phase 1 Verification: taskgraph validate (4 tasks OK), cargo fmt --check, cargo test, em-dash grep clean; architecture-reviewer pass returned zero critical issues
586 lines
30 KiB
Markdown
586 lines
30 KiB
Markdown
---
|
|
status: draft
|
|
last_updated: 2026-11-06
|
|
---
|
|
|
|
# alkgen: Phase 0 Research Findings
|
|
|
|
This document captures Phase 0 (Exploration) findings and open design
|
|
questions for the `alkgen` crate. The objective of Phase 0 per
|
|
`docs/sdd_process.md` is: *"Capture vision and guiding principles;
|
|
research options; validate approaches; converge on a recommended
|
|
approach."* It is the input to Phase 1 (Architecture).
|
|
|
|
Drafted 2026-09-08 from the initial setup discussion and a survey of
|
|
the relevant references (below). Updated 2026-11-06 at the unpause
|
|
(see §"The 2026-11-06 unpause (alkforge SSR)"). The crate is the
|
|
codegen engine of the alk* stack: a dev tool generating static files
|
|
into downstream projects from schema documents, i.e. handlebars-rust
|
|
templates rendered over alktype BAST documents (branch 1) or
|
|
jsonschema/openapi documents (branch 2), producing code artifacts.
|
|
Plus, since the unpause, plain HTML generation for alkforge's
|
|
server-side rendering (branch 0). Four branches; the branch numbering
|
|
below keeps the original order (branch 0 = the unpause addition, then
|
|
branches 1-3 in their historical order), so every "branch 1" / "branch
|
|
2" / "branch 3" reference in this doc and in the POC summary keeps its
|
|
original meaning:
|
|
|
|
0. **HTML generation for alkforge (SSR)** from 2026-11-06: alkforge's
|
|
web view (family ADR-002, the gitea replacement) needs server-side
|
|
HTML generation, and alkgen provides it: templates rendered over
|
|
page/data contexts, emitting HTML fragments or pages. Simpler than
|
|
the code branches and needed before them, so it jumps the queue as
|
|
the first real consumer.
|
|
1. **BAST readers/writers/validators**: from alktype BAST
|
|
documents, generate static binary read/write/validate code
|
|
integrating with alkcall's channels. The originally planned v1
|
|
target.
|
|
2. **Call-protocol handlers**: generate the boilerplate for
|
|
call-protocol (JSON RPC) integration, from jsonschema/openapi
|
|
(or BAST) inputs, i.e. the static counterpart of alkhttp's
|
|
`from_openapi` / `from_jsonschema` adapters. Second.
|
|
3. **Template-JIT core**: an alkminer-v1-shaped render-then-compile
|
|
cache with partials. Far down the timeline; influences only how
|
|
general the core template layer must be. Not a v1 requirement.
|
|
|
|
## The 2026-11-06 unpause (alkforge SSR)
|
|
|
|
The project paused 2026-09-08 with the reader POC complete. It
|
|
unpaused 2026-11-06 because alkforge arrived: family ADR-002
|
|
(`/workspace/@alkdev/alkfamily/docs/architecture/decisions/002-alkforge-gitea-replacement.md`)
|
|
records the decision to replace the self-hosted gitea deployment with
|
|
a family crate, and alkforge's web view needs server-side HTML
|
|
generation. alkforge files that ask upstream rather than forking
|
|
around it (ADR-002's no-deformation rule), so the SSR need flows into
|
|
alkgen's design.
|
|
|
|
What this adds, and what it changes about the framing:
|
|
|
|
- **The SSR branch is a runtime rendering use, not a dev-tool use.**
|
|
The code branches (1 and 2) run at a consumer's build time and emit
|
|
static files into its source tree (OQ-GN-01). The SSR branch means
|
|
alkgen is linked into alkforge as a library and renders HTML on
|
|
demand: same template/registry/context machinery, different
|
|
deployment posture. The settled answer for the code branches stands;
|
|
it just no longer covers the whole crate.
|
|
- **The core is a text-generation engine, target classes are
|
|
template sets.** HTML comes free (handlebars' native case), Rust
|
|
codegen is proven (reader POC), WGSL is proven (alkminer's kernel
|
|
builder); the shape is identical. The constraint with a real
|
|
consumer behind it: do not bake reader/writer assumptions into
|
|
shared core types (OQ-GN-09).
|
|
- **Queue order**: SSR first (alkforge needs it before the codegen
|
|
branches matter to it), then BAST readers (branch 1), then
|
|
call-protocol handlers (branch 2), then the template-JIT core
|
|
(branch 3). The reader POC findings carry over unchanged: they
|
|
constrain the core's strict-registry/partials/context shape, which
|
|
SSR shares.
|
|
- **Dependency refresh**: the unpause came after several months of
|
|
dependency drift. alktype is now 0.4.0 (reservation read/write API,
|
|
maxLength reserved encoding); the skeleton and the reader POC were
|
|
revalidated against it (see OQ resolution notes and
|
|
`reader-poc-summary.md`'s re-validation note).
|
|
|
|
### Core generalization steer (2026-11-06, setup discussion)
|
|
|
|
Principal steer after the unpause: the shared core should generalize
|
|
across three real targets, with the dev tool (static files into
|
|
downstream projects) demoted from "the product" to a nice-to-have
|
|
projection of the same machinery:
|
|
|
|
- **HTML generation (branch 0)**: the first and simplest customer.
|
|
Its value is structural: no plan projection, no layout math, no
|
|
typed-output machinery. If the core renders HTML cleanly without
|
|
target-specific exceptions, the codegen paths are not overfitted.
|
|
- **Rust codecs (branch 1)**: the reader POC already showed the
|
|
branch-specific complexity lives in the context builder (plan
|
|
projection, OQ-GN-02), not the registry/render layer. That is
|
|
evidence this split works.
|
|
- **WGSL (confirmed coming; branch 3 territory)**: alkminer-v1's
|
|
`ShaderBuilder` is the precedent, and the JS runtime will require
|
|
codegen that generates the WGSL. There is prior research in that
|
|
direction in the family. Because alktype + handlebars are already
|
|
wasm-clean, a wasm build can carry the render-then-compile cache
|
|
itself (render as the JIT step at runtime). Far down the timeline,
|
|
but it fixes one design requirement today: keep partials
|
|
first-class and keep rendering pure, synchronous, and deterministic
|
|
so the render step can be cached and compiled downstream. Retrofit
|
|
cost would be high later, preservation cost now is near zero.
|
|
|
|
Framing correction (2026-11-06, same discussion, second steer): the
|
|
"document generation vs code generation" split above carries a false
|
|
distinction. The core is a text-generation engine: contexts +
|
|
templates + partials → text. HTML is what handlebars was designed
|
|
for, Rust codecs are proven by the reader POC, and WGSL is proven by
|
|
alkminer's kernel builder; three data points, one shape. Neither the
|
|
registry/layout/render machinery nor the context interface changes
|
|
because the output text happens to be code: if the core can generate
|
|
valid HTML, it can generate valid code. What genuinely differs per
|
|
target is only:
|
|
|
|
- the template set and context shape (plan-driven projection vs page
|
|
data views vs shader contexts);
|
|
- the emit posture (static files at build time vs in-library render
|
|
vs render-then-compile feed);
|
|
- the validity pinning (each template set's render tests pin its
|
|
notion of valid output; the core cannot know what "valid" means
|
|
for WGSL vs HTML vs Rust and should not pretend to).
|
|
|
|
Concretely, the working split for Phase 1:
|
|
|
|
- **Shared core**: registry construction (strict mode, first-class
|
|
partials), context conventions, deterministic render entry point,
|
|
typed errors. Everything wasm-clean, synchronous, allocation-lean.
|
|
- **Per target**: the template set, the context shape, the emit
|
|
posture, and the validity pinning.
|
|
- **Demoted, not dead**: the dev-tool emit model (OQ-GN-01) stays the
|
|
shape of branches 1/2's output but is no longer the definition of
|
|
the crate; the crate is "text generation from templates and
|
|
partials over typed contexts", with targets as template sets plus
|
|
emit layers.
|
|
|
|
This refines OQ-GN-09: the SSR branch is not a special case to be
|
|
accommodated; it is the first instantiation and the generality test.
|
|
WGSL is the third instantiation and needs nothing from the core that
|
|
the first two do not already require.
|
|
|
|
## What is already settled
|
|
|
|
- **The stack shape**: handlebars-rust renders templates; alktype
|
|
provides the BAST document model (`Schema`/`Definitions` builder
|
|
types, `AlkTypeKind`, `OffsetMap`, `PackedLayout`, `read_plan`,
|
|
`validation_plan`); alkgen owns template assets, context building,
|
|
registry construction, and rendering. Output is static code files
|
|
generated into downstream projects (dev-tool model, OQ-GN-01) for
|
|
the code branches; in-library HTML rendering for the SSR branch
|
|
(see §unpause).
|
|
- **wasm-clean is validated, not assumed.** The skeleton (alktype
|
|
+ handlebars 6.4 + serde_json preserve_order + thiserror) compiles
|
|
to `wasm32-unknown-unknown` release as-is; re-verified 2026-11-06
|
|
with alktype 0.4.0. handlebars-rust ships its own wasm playground
|
|
(`wasm/` + `build-wasm.sh`), so keeping the constraint costs
|
|
nothing. Only the optional handlebars features are platform-bound
|
|
(`dir_source` → walkdir/fs, `script_helper` → rhai); the crate
|
|
avoids both.
|
|
- **The handlebars version**: latest stable is 6.4.4 (crates.io); the
|
|
vendored `/workspace/handlebars-rust` is head, exactly one commit
|
|
past the release tag. Pin `6.4` from crates.io; the vendored
|
|
checkout serves as reading material / emergency patch source only.
|
|
- **Strict mode is the default posture**: alkminer-v1's
|
|
`ShaderBuilder` (the closest prior art for a multi-template strict
|
|
registry) sets `set_strict_mode(true)`; for codegen, a missing
|
|
variable must be a render error, not silently-empty output (silent
|
|
gaps surface as compile bugs in the consumer's build, far from the
|
|
cause). This is AGENTS.md convention 6. For the SSR branch the same
|
|
posture applies (a missing variable in a page template is a render
|
|
error, not silently-blank HTML).
|
|
- **Templates are code**: inline `const &str` or `include_str!`,
|
|
versioned in-repo, render tests pinning golden output (typebox-rs
|
|
precedent, strengthened into AGENTS.md convention 7). Applies to
|
|
HTML templates too: a page render's golden output is pinned the
|
|
same way.
|
|
- **Determinism and hygiene**: generated artifacts carry the
|
|
"Auto-generated by alkgen. DO NOT EDIT." banner; same BAST + same
|
|
templates + same options → byte-identical output (AGENTS.md
|
|
convention 8). Note the interaction with `preserve_order`: alktype
|
|
depends on schema-JSON key order (packed byte order), so contexts
|
|
must preserve document order; sort only where a template
|
|
explicitly requires it. For the SSR branch the banner does not
|
|
apply to rendered HTML, but determinism still does (same BAST +
|
|
templates + options → byte-identical HTML; alkforge caching and
|
|
tests both lean on it).
|
|
|
|
## Prior art
|
|
|
|
### typebox-rs `src/codegen`, the direct ancestor
|
|
|
|
`/workspace/@alkimiadev/typebox-rs/src/codegen/rust.rs` is the older,
|
|
simpler version of this idea, and it validates the basic shape:
|
|
|
|
- `Handlebars<'static>` registry, templates registered from inline
|
|
`const &str`, per-schema-kind context structs (`#[derive(Serialize)]`)
|
|
built in Rust, one mapping function (`schema_to_rust_type`) turning
|
|
schema kinds into target-language type strings.
|
|
- Weaknesses to not inherit (it predates the alk* conventions):
|
|
templates registered with `.unwrap()` (lib convention: typed
|
|
errors), no strict mode (silently-empty output possible), no
|
|
generated-code banner on fragments, no determinism discipline
|
|
(`type_refs: HashMap` in context), template tests assert only
|
|
`contains(...)` rather than pinning golden output.
|
|
- What it lacks entirely: partials, layout awareness (it generated
|
|
serde-derive structs, not wire readers/writers), multi-branch
|
|
output (readers + writers + validators from one document).
|
|
|
|
### alkminer-v1 `compute/kernel.rs`: handlebars as JIT
|
|
|
|
`ShaderBuilder` = strict registry + `register_partial` /
|
|
`register_template` / `render`, followed by render-then-compile with
|
|
caching downstream. Branch 3 (the codegen JIT) is literally this
|
|
shape with compile targets other than wgpu pipelines. Structural
|
|
takeaway for the core layer: build the registry so partials are
|
|
first-class (registerable, composable) from the start; that is the
|
|
only branch-3 requirement that leaks back into v1, and it is cheap.
|
|
The same structure serves the SSR branch: alkforge's page templates
|
|
will want partials (header/footer/nav) as first-class registerable
|
|
assets.
|
|
|
|
### alktype, the interpreter that implies the compiler
|
|
|
|
alktype ships runtime plans (`OffsetMap`, `PackedLayout`,
|
|
`read_plan::ReadPlan`, `validation_plan::ValidationPlan`): it walks
|
|
BAST documents at runtime, per schema load. Branch 1 is the
|
|
*compile-time counterpart*: the same walks, but emitting code instead
|
|
of executing plans. The plans are therefore the best available spec
|
|
for what generated readers/writers/validators must do; they encode
|
|
the semantics (bounds checks, endianness, variable-length encoding,
|
|
TUnion dispatch, alignment) that templates must reproduce as literal
|
|
code. Key question thread: how much of the plan structure should
|
|
drive template context, and where does the interpreter/compiler
|
|
duplication get cut (see OQ-GN-02, OQ-GN-03).
|
|
|
|
### alkhttp adapters, the handler shape for branch 2
|
|
|
|
`from_openapi` / `from_jsonschema` build operation tables from
|
|
external schema documents. Branch 2 generates such adapters (or the
|
|
code they'd hand-write) from BAST/JSON documents. Farther out; mainly
|
|
a reminder that the core should not bake reader/writer assumptions
|
|
into shared types.
|
|
|
|
## Open Questions
|
|
|
|
Numbered OQ-<theme>-NN.. for reference and promotion to
|
|
`docs/architecture/open-questions.md` in Phase 1.
|
|
|
|
### OQ-GN-09 (new, 2026-11-06): What does the SSR branch require of the shared core?
|
|
|
|
alkforge needs server-side HTML generation before the code branches
|
|
matter to it (see §unpause). Opened by the unpause; this is the queue
|
|
entry point for Phase 1.
|
|
|
|
**Half-hunch (to be validated):** the shared core is the same in both
|
|
target classes, namely registry construction (strict mode, partials
|
|
as first-class assets), context building, and a rendering entry
|
|
point, all synchronous and wasm-clean (alkforge wants SSR without
|
|
platform surprises). What differs per target class is the template
|
|
set and the context shape: HTML contexts (page/data views, no layout
|
|
math, no plan projection) vs code contexts (plan-driven projection
|
|
per the reader POC). Candidates for what must be decided early:
|
|
|
|
- Library posture for SSR: alkgen in-process (linked into alkforge;
|
|
render-per-request or render-per-build) vs pre-rendered static
|
|
assets. In-process rendering with deterministic output enables
|
|
hashing/caching; pre-rendering reuses the dev-tool shape directly.
|
|
Affects alkgen's dependency weight inside alkforge (wasm-clean
|
|
matters less in-process; strict sync API matters everywhere).
|
|
- Output unit for HTML: full page vs fragment vs both (partials make
|
|
fragments natural; OQ-GN-01's fragment exclusion was scoped to the
|
|
code branches).
|
|
- Whether the DO-NOT-EDIT banner and generated-code hygiene rules
|
|
apply (they were written for code artifacts; rendered HTML is a
|
|
runtime consumer product, not checked-in generated code, unless
|
|
alkforge pre-renders).
|
|
- Error posture in the SSR path: render failure → typed error →
|
|
alkforge's error page; no panics (same convention 2 rationale:
|
|
alkforge serves arbitrary git/HTTP traffic).
|
|
|
|
Residue for Phase 1: the exact shared-core split (what is
|
|
template-set-agnostic vs per-target), and the alkforge SSR
|
|
library/API shape.
|
|
|
|
### OQ-GN-01: What is alkgen's primary unit of output: a crate, a module, a fragment?
|
|
|
|
**Resolved 2026-09-08 (setup discussion): static files generated into
|
|
a consumer project.** The clarification that resolved it: alkcall is
|
|
a *library* others build on, and branches 1 and 2 are a
|
|
*dev tool* consuming it; it generates boilerplate into some downstream
|
|
project
|
|
(protocol crates like alktty/alktunnels/sftp, cloud-provider
|
|
integrations like vast.ai/runpod), not into alkcall itself. Both
|
|
branches take a provided schema document (BAST or jsonschema/
|
|
openapi) and emit static files.
|
|
|
|
- **Fragment** is out; the consumer assembling fragments by hand
|
|
defeats the dev-tool purpose.
|
|
- **Crate** is out for v1; generating a whole Cargo project is
|
|
trivially composed from modules later if a consumer asks, and
|
|
would bake build-layout decisions (workspace placement, dep
|
|
versions) into the generator prematurely.
|
|
- **Module-level output is the v1 shape**: one document → one or
|
|
more self-contained `.rs` files (banner + use statements +
|
|
impls), dropped into the consumer's `src/` tree. Matches
|
|
typebox-rs `generate_module` and how generated protocol files
|
|
would sit in e.g. an alktunnels-style crate.
|
|
|
|
2026-11-06 scope note: the fragment exclusion was written for the
|
|
code branches. The SSR branch (OQ-GN-09) may want fragment-level
|
|
HTML output; that question does not reopen this resolution.
|
|
|
|
Residue for Phase 1: file partitioning policy (one file per
|
|
document vs per-type files; where the reader/writer/validator
|
|
split lands), and whether generated files are regenerated wholesale
|
|
(overwrite) or support partial regeneration. These are spec-shape
|
|
tasks, not research questions.
|
|
|
|
### OQ-GN-02: Interpretation depth: render from BAST directly, or from alktype's plans?
|
|
|
|
Templates need a context. Two candidate sources:
|
|
|
|
- **BAST document** (`serde_json::Value` schema + `Definitions`):
|
|
closest to the source of truth; templates/contexts do their own
|
|
walking (typebox-rs did this).
|
|
- **alktype plans** (`OffsetMap`, `ReadPlan`, `ValidationPlan`):
|
|
pre-digested semantics; offsets, sizes, bounds-check shapes,
|
|
dispatch structure are already computed and validated; the
|
|
template context becomes a projection of the plan rather than a
|
|
re-derivation.
|
|
|
|
The plan-driven path risks baking alktype internals into template
|
|
contexts (plans are public API but evolve); the BAST-driven path
|
|
re-duplicates the layout math alktype already owns (drift risk:
|
|
generated readers disagreeing with the engine's own
|
|
interpretation). Half-hunch: plans as the semantic source for
|
|
offsets/sizes, BAST for descriptive metadata (names, docs), with
|
|
the exact projection decided in Phase 1. This decision shapes the
|
|
whole context-builder layer.
|
|
|
|
**Direction set 2026-09-08 (setup discussion): plan-driven, with BAST
|
|
for descriptive metadata.** Supporting rationale:
|
|
|
|
- The layout math (offsets, sizes, dispatch structure) is already
|
|
computed *and validated* by the engine; re-deriving it in the
|
|
template layer would reintroduce exactly the drift OQ-GN-03
|
|
flags as the standing risk, by construction.
|
|
- alktype is already a dependency; plan computation at generation
|
|
time costs nothing extra.
|
|
- The residual risk inverts: instead of drifting from engine
|
|
semantics, contexts now track alktype's public plan API. If a
|
|
plan shape evolves, context builders evolve with it, an
|
|
ordinary dependency-upgrade concern, not a correctness risk.
|
|
|
|
**Refined by the reader POC (finding F-6):** for typed output, names
|
|
come from the BAST doc (the plan eagerly resolves `$ref`s and loses
|
|
def names), while the plan remains the semantic source. Net: plan
|
|
for semantics, BAST doc for names.
|
|
|
|
Residue for Phase 1 (spec-shape, not research): the exact projection
|
|
(which plan structures (`OffsetMap` entries, `ReadPlan` nodes,
|
|
`ValidationPlan` fields) map to which template-context shapes) and
|
|
where BAST metadata (names, docs, annotations) joins the projection.
|
|
This is the context-builder layer's spec task.
|
|
|
|
### OQ-GN-03: Where does the interpreter/compiler duplication get cut?
|
|
|
|
**Resolved as posture, 2026-09-08 (setup discussion): the duplication
|
|
is the point; two deployment profiles, complements not competitors.**
|
|
|
|
- **alktype (interpreter)** serves the dynamic case: schemas arrive at
|
|
runtime from arbitrary peers (alkcall hub/spoke, binary rpc); the
|
|
plan-walk cost is the price of not trusting the schema. It is near
|
|
the performance floor for a schema-driven method, but sits roughly
|
|
20x worse than hand-rolled code in several paths.
|
|
- **alkgen (compiler)** serves the static case: schema known at build
|
|
time, dispatch compiled to native code. The ~20x gap is the value
|
|
proposition; recovering most of it is branch 1's success metric
|
|
(OQ-GN-08's three-way benchmark). "Static is faster" applies to
|
|
branch 2 the same way: generated handlers with dispatch compiled
|
|
in vs alkhttp adapters interpreting a spec document at runtime.
|
|
- This is also why the crate starts now, before the JIT-ish branch 3:
|
|
the static path already has a consumer (alkcall channels).
|
|
|
|
The reader POC sharpened this: at the `Value`-materializing output
|
|
contract the generated path is already near the hand-written ceiling
|
|
(only ~10-20% below); the interpreter win is ~1.9x, and the 20x-scale
|
|
gains live at the typed output contract (resolved for branch 1 by the
|
|
POC's second render, typed direct codecs).
|
|
|
|
Residue (kept open): the semantic single-source question. The plans
|
|
(`ReadPlan`, `ValidationPlan`, `OffsetMap`) remain the spec generated
|
|
code must mirror; engine/generated-code drift is the standing risk,
|
|
tracked via OQ-GN-02 (context source). "alktype consumes alkgen's
|
|
tables" is not on the table for v1; revisit only if drift bites.
|
|
|
|
### OQ-GN-04: Target-language scope for v1
|
|
|
|
Branch 1 output is Rust (alkcall is Rust). Does alkgen need a
|
|
target-language abstraction (template sets per language, typebox-rs
|
|
had `rust.rs` + `typescript.rs`) from day one, or is "Rust-only
|
|
templates" acceptable with the core kept language-agnostic? Half-
|
|
hunch: keep the core (context building, registry, rendering) target-
|
|
agnostic; make v1 templates Rust-only; defer any TS/WASM template
|
|
set until a consumer asks. Cost of retrofit is low if contexts stay
|
|
language-neutral. The 2026-11-06 unpause adds a non-Rust-output data
|
|
point for this hunch: the SSR branch emits HTML with the same core,
|
|
which is exactly the "contexts stay language-neutral, template sets
|
|
differ" pattern working out in practice.
|
|
|
|
### OQ-GN-05: Reader/writer code style: raw bytes or alktype-delegating?
|
|
|
|
Generated readers can be (a) self-contained byte manipulation
|
|
(offsets + `as` casts + bounds checks, fully compiled), (b) thin
|
|
callers into alktype's runtime (`SequentialReader` with plans), or
|
|
(c) a middle shape (compile-time offset tables, runtime calls for
|
|
variable-length/union paths). (a) maximizes compile-time value but
|
|
duplicates the most semantics; (b) is nearly pointless (just calls
|
|
alktype); (c) is the likely landing zone but the boundary is
|
|
undecided. Interacts with OQ-GN-02/03. Needs a POC-quality answer
|
|
before Phase 1 ADR.
|
|
|
|
**Resolved 2026-09-08 (reader POC, see `reader-poc-summary.md`):**
|
|
option (a), self-contained generated code. The POC built a reader
|
|
with zero alktype runtime dependency, byte-parity-exact against the
|
|
interpreter, ~1.9x faster. Uniform cursor addressing (not source
|
|
offset folding) is the right model; LLVM folds constant cursor
|
|
chains (finding F-1). The (c) middle shape is moot: compiled
|
|
dispatch wins without runtime delegation.
|
|
|
|
### OQ-GN-06: Does branch 1 output depend on alktype at all?
|
|
|
|
**Resolved 2026-09-08 (reader POC):** generated output depends only
|
|
on `serde_json` (the `Value` output contract). alktype appears only
|
|
in the generator (build script); consumers' generated code is
|
|
alktype-free. The POC's second render (typed output) went further:
|
|
zero dependencies at all, not even serde_json. Caveat carried to
|
|
Phase 1: if the output contract changes shape again, the dependency
|
|
list moves with it (typed with serde bridges would reintroduce a
|
|
serde derive dep; see the POC summary's (a) vs (b) resolution).
|
|
|
|
Original framing (retained for context): generated files land in
|
|
*downstream projects* which do not currently depend on alktype, so
|
|
a self-contained (a) output makes the generated module
|
|
dependency-free, while a delegating (b/c) output adds alktype as a
|
|
runtime dep of the consumer project.
|
|
|
|
### OQ-GN-07: Validation strategy in generated code
|
|
|
|
alktype's `validate_bytes` materializes a `Value` from bytes then
|
|
runs jsonschema, a heavyweight runtime path. Generated validators
|
|
could instead emit checks derived from `validation_plan` (no
|
|
materialization, no jsonschema dep in generated code), but then
|
|
generated validation must track alktype's validation semantics
|
|
exactly (drift risk, same as OQ-GN-02). Unresolved; affects
|
|
whether generated code needs a jsonschema dependency.
|
|
|
|
**Partially resolved 2026-09-08 (reader POC):** the read path's
|
|
validation (bounds checks, UTF-8 validation, union dispatch limits)
|
|
is proven generatable as error-returning code with no jsonschema
|
|
dependency. The full BAST-validation question (jsonschema-derived
|
|
semantic checks) remains open; see the POC summary's follow-up #5.
|
|
|
|
### OQ-GN-08: POC scope
|
|
|
|
Candidate POCs (in rough priority order), per the SDD "validate
|
|
promising approaches":
|
|
|
|
1. **Reader POC (OQ-GN-05/06/07)**: take one small BAST document
|
|
(a flat struct + one variable-length field + one TUnion), render
|
|
a reader via handlebars from an `OffsetMap`/`ReadPlan`-derived
|
|
context, compile it, and check byte-level agreement against
|
|
alktype's own `SequentialReader` on a fixed corpus. This answers
|
|
the biggest cluster (context source, delegation boundary,
|
|
dependency question) in one executable artifact.
|
|
2. **Strict-registry + partials core POC**: cheap; validates the
|
|
alkminer-shaped core against branch-1 needs (probably just
|
|
folded into POC 1).
|
|
3. **Handler-adapter POC (branch 2)**: deferred until branch 1
|
|
converges.
|
|
|
|
**Resolved 2026-09-08 (reader POC, see `reader-poc-summary.md`):**
|
|
POC 1 ran as the standalone-crate precedent (`/workspace/alkgen-
|
|
reader-poc`, self-contained, findings written back here). Results:
|
|
plan-driven handlebars codegen validated end-to-end; parity-exact,
|
|
~1.9x faster than the interpreter, and a decomposition finding:
|
|
the ~20x-vs-hand-rolled gap is `Value`-allocation cost, not dispatch
|
|
overhead (output-parity hand-rolled baseline is only ~10-20% faster
|
|
than generated). Biggest Phase 1 ADR: the output contract
|
|
(Value-materializing vs typed structs), resolved for branch 1 by the
|
|
typed second render (direct codecs). POC 2 folded into POC 1
|
|
(strict registry + partials exercised); POC 3 deferred.
|
|
|
|
**Calibration input, 2026-09-08 (setup discussion):** the interpreter
|
|
sits ~20x worse than hand-rolled code on several paths; branch 1's
|
|
success metric is recovering most of that gap. POC 1 should add a
|
|
three-way benchmark (alktype plan-walk vs generated reader vs a
|
|
hand-rolled baseline); it doubles as the drift check *and* the
|
|
performance validation, and gives the "how much of the 20x do we
|
|
recover" answer with numbers instead of vibes. This raises POC 1's
|
|
value: the delegation boundary (OQ-GN-05) is now a performance
|
|
trade, not just a structural one; every runtime call into alktype
|
|
is a place the compiled path stays interpreter-shaped.
|
|
|
|
## Survey / prior-art list
|
|
|
|
- `/workspace/@alkimiadev/typebox-rs/src/codegen/rust.rs`: context
|
|
structs, inline templates, per-kind type mapping (§Prior art).
|
|
- `/workspace/@alkminer/reference/alkminer-v1/src/compute/kernel.rs`:
|
|
strict registry, partials-first core, render-then-compile (§
|
|
Prior art).
|
|
- `/workspace/@alkdev/alktype/src/`: `read_plan.rs`,
|
|
`validation_plan.rs`, `offset_map.rs`, `layout_builder.rs`; the
|
|
runtime semantics generated code must mirror. `docs/architecture/`
|
|
there (layout-engine.md, data-access.md, validation.md) is the
|
|
authoritative behavior spec. Now at 0.4.0.
|
|
- `/workspace/@alkdev/alkhttp/src/adapters/from_openapi.rs` /
|
|
`from_jsonschema.rs`: the handler shape branch 2 targets.
|
|
- `/workspace/@alkdev/alkcall`: the consumer for branch 1 output
|
|
(channels readers/writers/validators); its AGENTS.md encodes the
|
|
no-panics/no-unsafe posture generated code must satisfy.
|
|
- `/workspace/@alkdev/alkforge` (docs-stage) + family ADR-002:
|
|
`/workspace/@alkdev/alkfamily/docs/architecture/decisions/002-alkforge-gitea-replacement.md`;
|
|
the SSR consumer that drove the unpause; its no-deformation rule
|
|
routes its asks into alkgen's design (OQ-GN-09).
|
|
- `/workspace/handlebars-rust`: vendored head checkout (1 commit
|
|
past v6.4.4); crates.io `6.4` is the pinned dep. wasm playground
|
|
under `wasm/` proves the wasm-clean story.
|
|
- `/workspace/alkgen-reader-poc`: the reader POC crate (2026-09-08;
|
|
re-validated on alktype 0.4.0, 2026-11-06): plan→context→render→
|
|
parity→bench end-to-end; `docs/research/reader-poc-summary.md`
|
|
here carries the findings.
|
|
|
|
## Convergence checklist (what Phase 0 must produce)
|
|
|
|
- [x] Stack validated: skeleton builds wasm-clean (alktype 0.3.0 +
|
|
handlebars 6.4, 2026-09-08; re-verified on alktype 0.4.0,
|
|
2026-11-06)
|
|
- [x] Prior art surveyed: typebox-rs, alkminer-v1, alktype plans,
|
|
alkhttp adapters (§Prior art)
|
|
- [x] Branch scope framed: initially readers/writers/validators
|
|
first, handlers second, JIT later (§opening); reshaped
|
|
2026-11-06 by the unpause: alkforge SSR (branch 0) jumps the
|
|
queue
|
|
- [x] OQ-GN-01 (output unit): resolved 2026-09-08: static files
|
|
generated into downstream projects (dev-tool model); module-
|
|
level output for v1, fragments and crate-generation out.
|
|
Residue: file partitioning policy (Phase 1 spec task).
|
|
2026-11-06 note: fragment exclusion is code-branch-scoped;
|
|
see OQ-GN-09 for the SSR branch's output unit.
|
|
- [x] OQ-GN-02/03 (context source, duplication cut): direction set
|
|
2026-09-08: plan-driven contexts (plans for offsets/sizes, BAST
|
|
for descriptive metadata); duplication resolved as posture;
|
|
refined by the POC (names from BAST, typed output resolved to
|
|
direct codecs).
|
|
Residue: exact plan→context projection (Phase 1 spec task)
|
|
- [x] OQ-GN-04 (language scope): dispositioned 2026-11-06 at Phase 1
|
|
start: core language-agnostic by construction (architecture
|
|
ADR-001/ADR-004), template sets written per consumer need (SSR
|
|
HTML first, Rust codecs second, WGSL deferred); tracker entry
|
|
OQ-08 in `docs/architecture/open-questions.md` records the
|
|
adoption. The SSR branch arriving with no core change is the
|
|
hunch's predicted data point
|
|
- [x] OQ-GN-05/06 (delegation boundary, dependency shape): resolved
|
|
2026-09-08 by reader POC: self-contained generated code,
|
|
serde_json-only dependency for the Value contract, zero-dep for
|
|
the typed contract; uniform cursor addressing (F-1)
|
|
- [ ] OQ-GN-07 (validation strategy): read-path validation proven
|
|
generatable; full BAST-validation strategy still open
|
|
- [x] OQ-GN-08 (POC decision): DONE 2026-09-08:
|
|
`reader-poc-summary.md`; three-way benchmark + parity corpus;
|
|
Phase 1 input list in the POC summary's follow-ups. Re-validated
|
|
2026-11-06 on alktype 0.4.0 (tests + bench unchanged)
|
|
- [ ] OQ-GN-09 (SSR branch scope, opened 2026-11-06): needs a
|
|
scope/library-posture answer (in-process rendering vs
|
|
pre-rendered, shared-core split) before Phase 1's ADRs; alkforge
|
|
is the consumer waiting on it. Refined by the core
|
|
generalization steer (§"Core generalization steer"): the core
|
|
split proposal is recorded; the remaining decision is the SSR
|
|
library posture and the exact core/emit boundary
|